Oct 30 19:05:25.454469 ip-10-0-137-200 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 19:05:25.847531 ip-10-0-137-200 kubenswrapper[2586]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:05:25.847531 ip-10-0-137-200 kubenswrapper[2586]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 19:05:25.847531 ip-10-0-137-200 kubenswrapper[2586]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:05:25.847531 ip-10-0-137-200 kubenswrapper[2586]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:05:25.852890 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852808    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:05:25.852890 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852888    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:05:25.852890 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852892    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:05:25.852890 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852896    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:05:25.853047 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852900    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:05:25.853047 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852903    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:05:25.853047 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852906    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:05:25.853047 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852908    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:05:25.853047 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852912    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:05:25.853047 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852914    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:05:25.853047 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852917    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:05:25.853047 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852920    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:05:25.853047 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852923    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:05:25.853047 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852925    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:05:25.853047 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852928    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:05:25.853047 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852931    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:05:25.853047 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852934    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:05:25.853047 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852936    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:05:25.853047 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852939    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:05:25.853047 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852941    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:05:25.853047 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852945    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:05:25.853047 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852947    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:05:25.853047 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852950    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:05:25.853508 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852953    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:05:25.853508 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852955    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:05:25.853508 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852958    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:05:25.853508 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852960    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:05:25.853508 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852963    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:05:25.853508 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852966    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:05:25.853508 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852969    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:05:25.853508 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852971    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:05:25.853508 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852982    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:05:25.853508 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852985    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:05:25.853508 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852987    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:05:25.853508 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852990    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:05:25.853508 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852993    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:05:25.853508 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.852996    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:05:25.853508 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853000    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:05:25.853508 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853003    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:05:25.853508 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853006    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:05:25.853508 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853009    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:05:25.853508 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853012    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:05:25.853955 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853015    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:05:25.853955 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853017    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:05:25.853955 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853020    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:05:25.853955 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853023    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:05:25.853955 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853025    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:05:25.853955 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853028    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:05:25.853955 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853030    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:05:25.853955 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853033    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:05:25.853955 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853037    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:05:25.853955 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853039    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:05:25.853955 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853042    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:05:25.853955 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853045    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:05:25.853955 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853047    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:05:25.853955 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853050    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:05:25.853955 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853054    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:05:25.853955 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853057    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:05:25.853955 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853059    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:05:25.853955 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853062    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:05:25.853955 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853064    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:05:25.853955 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853067    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:05:25.854433 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853070    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:05:25.854433 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853072    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:05:25.854433 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853075    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:05:25.854433 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853077    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:05:25.854433 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853080    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:05:25.854433 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853083    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:05:25.854433 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853087    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:05:25.854433 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853092    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:05:25.854433 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853094    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:05:25.854433 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853097    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:05:25.854433 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853100    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:05:25.854433 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853102    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:05:25.854433 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853105    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:05:25.854433 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853107    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:05:25.854433 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853110    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:05:25.854433 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853113    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:05:25.854433 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853115    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:05:25.854433 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853118    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:05:25.854433 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853121    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:05:25.854884 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853123    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:05:25.854884 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853126    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:05:25.854884 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853129    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:05:25.854884 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853131    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:05:25.854884 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853134    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:05:25.854884 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853136    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:05:25.854884 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853139    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:05:25.854884 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853142    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:05:25.854884 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853144    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:05:25.854884 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853147    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:05:25.854884 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853149    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:05:25.854884 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853152    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:05:25.854884 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853154    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:05:25.854884 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853157    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:05:25.854884 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853160    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:05:25.854884 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853162    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:05:25.854884 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853165    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:05:25.854884 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853167    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:05:25.854884 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853170    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:05:25.855369 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853173    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:05:25.855369 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853178    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:05:25.855369 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853183    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:05:25.855369 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853186    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:05:25.855369 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853189    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:05:25.855369 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853191    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:05:25.855369 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853985    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:05:25.855369 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853993    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:05:25.855369 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.853997    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:05:25.855369 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854000    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:05:25.855369 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854003    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:05:25.855369 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854006    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:05:25.855369 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854008    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:05:25.855369 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854011    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:05:25.855369 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854014    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:05:25.855369 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854016    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:05:25.855369 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854024    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:05:25.855369 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854027    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:05:25.855369 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854029    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:05:25.855897 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854032    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:05:25.855897 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854035    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:05:25.855897 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854037    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:05:25.855897 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854040    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:05:25.855897 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854042    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:05:25.855897 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854045    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:05:25.855897 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854048    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:05:25.855897 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854050    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:05:25.855897 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854053    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:05:25.855897 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854056    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:05:25.855897 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854058    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:05:25.855897 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854061    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:05:25.855897 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854063    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:05:25.855897 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854066    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:05:25.855897 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854070    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:05:25.855897 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854073    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:05:25.855897 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854076    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:05:25.855897 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854078    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:05:25.855897 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854081    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:05:25.855897 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854084    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:05:25.856402 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854087    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:05:25.856402 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854090    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:05:25.856402 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854093    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:05:25.856402 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854095    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:05:25.856402 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854098    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:05:25.856402 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854101    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:05:25.856402 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854104    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:05:25.856402 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854106    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:05:25.856402 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854109    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:05:25.856402 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854112    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:05:25.856402 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854115    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:05:25.856402 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854117    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:05:25.856402 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854120    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:05:25.856402 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854122    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:05:25.856402 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854125    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:05:25.856402 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854127    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:05:25.856402 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854130    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:05:25.856402 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854132    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:05:25.856402 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854135    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:05:25.856857 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854138    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:05:25.856857 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854140    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:05:25.856857 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854143    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:05:25.856857 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854146    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:05:25.856857 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854148    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:05:25.856857 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854151    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:05:25.856857 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854153    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:05:25.856857 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854157    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:05:25.856857 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854159    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:05:25.856857 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854162    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:05:25.856857 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854164    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:05:25.856857 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854167    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:05:25.856857 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854170    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:05:25.856857 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854172    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:05:25.856857 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854175    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:05:25.856857 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854177    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:05:25.856857 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854181    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:05:25.856857 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854184    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:05:25.856857 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854186    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:05:25.857311 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854190    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:05:25.857311 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854194    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:05:25.857311 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854197    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:05:25.857311 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854200    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:05:25.857311 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854205    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:05:25.857311 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854209    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:05:25.857311 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854227    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:05:25.857311 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854229    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:05:25.857311 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854232    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:05:25.857311 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854236    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:05:25.857311 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854239    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:05:25.857311 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854242    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:05:25.857311 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854245    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:05:25.857311 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854248    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:05:25.857311 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854251    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:05:25.857311 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854253    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:05:25.857311 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854256    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:05:25.857311 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854259    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:05:25.857311 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854261    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:05:25.857768 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854264    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:05:25.857768 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854266    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:05:25.857768 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854269    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:05:25.857768 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854272    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:05:25.857768 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854274    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:05:25.857768 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854277    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:05:25.857768 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854280    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:05:25.857768 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854284    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:05:25.857768 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854286    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:05:25.857768 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854289    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:05:25.857768 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854291    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:05:25.857768 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854294    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:05:25.857768 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854296    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:05:25.857768 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854299    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:05:25.857768 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854302    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:05:25.857768 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854304    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:05:25.857768 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854836    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:05:25.857768 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854843    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:05:25.857768 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854846    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:05:25.858243 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854851    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:05:25.858243 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854855    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:05:25.858243 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854858    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:05:25.858243 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854861    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:05:25.858243 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854864    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:05:25.858243 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854867    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:05:25.858243 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854869    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:05:25.858243 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854872    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:05:25.858243 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854875    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:05:25.858243 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854877    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:05:25.858243 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854880    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:05:25.858243 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854883    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:05:25.858243 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854885    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:05:25.858243 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854888    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:05:25.858243 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854891    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:05:25.858243 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854894    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:05:25.858243 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854897    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:05:25.858243 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854900    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:05:25.858243 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854902    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:05:25.858698 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854905    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:05:25.858698 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854908    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:05:25.858698 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854910    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:05:25.858698 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854913    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:05:25.858698 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854915    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:05:25.858698 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854918    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:05:25.858698 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854921    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:05:25.858698 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854923    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:05:25.858698 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854926    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:05:25.858698 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854928    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:05:25.858698 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854931    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:05:25.858698 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854934    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:05:25.858698 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854936    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:05:25.858698 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854939    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:05:25.858698 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854942    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:05:25.858698 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854945    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:05:25.858698 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854948    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:05:25.858698 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854951    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:05:25.858698 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854953    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:05:25.858698 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854956    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:05:25.859185 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854958    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:05:25.859185 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854961    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:05:25.859185 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854965    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:05:25.859185 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854968    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:05:25.859185 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854971    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:05:25.859185 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854974    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:05:25.859185 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854977    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:05:25.859185 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854980    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:05:25.859185 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854982    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:05:25.859185 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854985    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:05:25.859185 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854988    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:05:25.859185 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854990    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:05:25.859185 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854993    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:05:25.859185 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854995    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:05:25.859185 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.854998    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:05:25.859185 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855001    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:05:25.859185 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855003    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:05:25.859185 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855006    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:05:25.859185 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855008    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:05:25.859639 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855011    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:05:25.859639 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855014    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:05:25.859639 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855017    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:05:25.859639 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855020    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:05:25.859639 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855023    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:05:25.859639 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855025    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:05:25.859639 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855028    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:05:25.859639 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855034    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:05:25.859639 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855036    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:05:25.859639 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855039    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:05:25.859639 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855042    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:05:25.859639 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855044    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:05:25.859639 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855047    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:05:25.859639 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855050    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:05:25.859639 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855053    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:05:25.859639 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855055    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:05:25.859639 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855058    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:05:25.859639 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855062    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:05:25.859639 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855065    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:05:25.859639 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855068    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:05:25.860109 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855070    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:05:25.860109 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855073    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:05:25.860109 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855075    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:05:25.860109 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855078    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:05:25.860109 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855081    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:05:25.860109 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855083    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:05:25.860109 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855086    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:05:25.860109 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855088    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:05:25.860109 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855091    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:05:25.860109 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855094    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:05:25.860109 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855096    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:05:25.860109 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855099    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:05:25.860109 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855102    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:05:25.860109 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855105    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:05:25.860109 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855107    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:05:25.860109 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855110    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:05:25.860109 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855114    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:05:25.860109 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855117    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:05:25.860109 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855120    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:05:25.860109 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855122    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855125    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855128    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855130    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855132    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.855135    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856097    2586 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856107    2586 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856113    2586 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856118    2586 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856123    2586 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856126    2586 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856130    2586 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856135    2586 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856138    2586 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856141    2586 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856145    2586 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856148    2586 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856151    2586 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856154    2586 flags.go:64] FLAG: --cgroup-root=""
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856157    2586 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856160    2586 flags.go:64] FLAG: --client-ca-file=""
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856163    2586 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 19:05:25.860615 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856166    2586 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856169    2586 flags.go:64] FLAG: --cluster-domain=""
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856172    2586 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856175    2586 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856178    2586 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856181    2586 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856187    2586 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856190    2586 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856193    2586 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856196    2586 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856199    2586 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856203    2586 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856206    2586 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856209    2586 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856224    2586 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856229    2586 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856232    2586 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856235    2586 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856238    2586 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856241    2586 flags.go:64] FLAG: --enable-server="true"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856244    2586 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856249    2586 flags.go:64] FLAG: --event-burst="100"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856252    2586 flags.go:64] FLAG: --event-qps="50"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856255    2586 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 19:05:25.861146 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856258    2586 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856261    2586 flags.go:64] FLAG: --eviction-hard=""
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856265    2586 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856268    2586 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856271    2586 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856274    2586 flags.go:64] FLAG: --eviction-soft=""
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856277    2586 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856280    2586 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856283    2586 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856286    2586 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856289    2586 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856292    2586 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856295    2586 flags.go:64] FLAG: --feature-gates=""
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856299    2586 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856303    2586 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856306    2586 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856310    2586 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856313    2586 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856316    2586 flags.go:64] FLAG: --help="false"
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856318    2586 flags.go:64] FLAG: --hostname-override="ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856322    2586 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856325    2586 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856328    2586 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 19:05:25.861772 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856331    2586 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856334    2586 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856337    2586 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856340    2586 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856343    2586 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856346    2586 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856349    2586 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856352    2586 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856355    2586 flags.go:64] FLAG: --kube-reserved=""
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856358    2586 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856361    2586 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856364    2586 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856367    2586 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856370    2586 flags.go:64] FLAG: --lock-file=""
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856373    2586 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856375    2586 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856378    2586 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856384    2586 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856387    2586 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856389    2586 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856392    2586 flags.go:64] FLAG: --logging-format="text"
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856395    2586 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856398    2586 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856402    2586 flags.go:64] FLAG: --manifest-url=""
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856406    2586 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 19:05:25.862733 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856410    2586 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856413    2586 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856417    2586 flags.go:64] FLAG: --max-pods="110"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856420    2586 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856424    2586 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856427    2586 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856430    2586 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856433    2586 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856435    2586 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856438    2586 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856445    2586 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856448    2586 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856451    2586 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856454    2586 flags.go:64] FLAG: --pod-cidr=""
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856457    2586 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856460    2586 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856463    2586 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856466    2586 flags.go:64] FLAG: --port="10250"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856469    2586 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856472    2586 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-0d92dc2b650bcfcd5"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856475    2586 flags.go:64] FLAG: --qos-reserved=""
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856478    2586 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856480    2586 flags.go:64] FLAG: --register-node="true"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856483    2586 flags.go:64] FLAG: --register-with-taints=""
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856487    2586 flags.go:64] FLAG: --registry-burst="10"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856490    2586 flags.go:64] FLAG: --registry-qps="5"
Oct 30 19:05:25.863807 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856493    2586 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856496    2586 flags.go:64] FLAG: --reserved-memory=""
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856499    2586 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856502    2586 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856506    2586 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856509    2586 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856512    2586 flags.go:64] FLAG: --runonce="false"
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856515    2586 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856518    2586 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856521    2586 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856524    2586 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856527    2586 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856530    2586 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856533    2586 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856536    2586 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856539    2586 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856542    2586 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856546    2586 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856549    2586 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856553    2586 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856555    2586 flags.go:64] FLAG: --system-cgroups=""
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856558    2586 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856563    2586 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856566    2586 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856569    2586 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 19:05:25.864919 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856573    2586 flags.go:64] FLAG: --tls-min-version=""
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856575    2586 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856578    2586 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856581    2586 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856584    2586 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856587    2586 flags.go:64] FLAG: --v="2"
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856591    2586 flags.go:64] FLAG: --version="false"
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856595    2586 flags.go:64] FLAG: --vmodule=""
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856599    2586 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.856602    2586 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856731    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856736    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856739    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856741    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856744    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856747    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856749    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856752    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856755    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856758    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856761    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856764    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:05:25.866005 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856767    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:05:25.866878 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856769    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:05:25.866878 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856774    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:05:25.866878 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856777    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:05:25.866878 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856779    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:05:25.866878 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856782    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:05:25.866878 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856785    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:05:25.866878 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856787    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:05:25.866878 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856790    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:05:25.866878 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856793    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:05:25.866878 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856795    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:05:25.866878 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856798    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:05:25.866878 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856801    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:05:25.866878 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856803    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:05:25.866878 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856806    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:05:25.866878 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856809    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:05:25.866878 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856812    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:05:25.866878 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856814    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:05:25.866878 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856817    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:05:25.866878 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856819    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:05:25.867562 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856822    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:05:25.867562 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856826    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:05:25.867562 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856829    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:05:25.867562 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856831    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:05:25.867562 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856834    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:05:25.867562 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856836    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:05:25.867562 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856839    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:05:25.867562 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856842    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:05:25.867562 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856844    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:05:25.867562 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856847    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:05:25.867562 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856850    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:05:25.867562 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856853    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:05:25.867562 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856856    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:05:25.867562 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856858    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:05:25.867562 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856862    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:05:25.867562 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856865    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:05:25.867562 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856868    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:05:25.867562 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856871    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:05:25.867562 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856873    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:05:25.867562 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856876    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:05:25.868380 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856879    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:05:25.868380 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856881    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:05:25.868380 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856884    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:05:25.868380 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856887    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:05:25.868380 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856889    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:05:25.868380 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856892    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:05:25.868380 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856895    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:05:25.868380 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856897    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:05:25.868380 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856900    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:05:25.868380 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856903    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:05:25.868380 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856905    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:05:25.868380 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856908    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:05:25.868380 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856911    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:05:25.868380 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856914    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:05:25.868380 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856918    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:05:25.868380 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856922    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:05:25.868380 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856925    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:05:25.868380 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856928    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:05:25.868380 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856931    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:05:25.868965 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856933    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:05:25.868965 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856936    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:05:25.868965 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856938    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:05:25.868965 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856941    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:05:25.868965 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856943    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:05:25.868965 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856946    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:05:25.868965 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856949    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:05:25.868965 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856953    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:05:25.868965 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856956    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:05:25.868965 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856958    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:05:25.868965 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856963    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:05:25.868965 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856967    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:05:25.868965 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856970    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:05:25.868965 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856973    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:05:25.868965 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856976    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:05:25.868965 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856979    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:05:25.868965 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856981    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:05:25.868965 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856984    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:05:25.868965 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856986    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:05:25.869620 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856989    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:05:25.869620 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856992    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:05:25.869620 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856995    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:05:25.869620 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.856997    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:05:25.869620 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857000    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:05:25.869620 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857002    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:05:25.869620 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857005    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:05:25.869620 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857008    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:05:25.869620 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857011    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:05:25.869620 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857014    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:05:25.869620 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857016    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:05:25.869620 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857019    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:05:25.869620 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857021    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:05:25.869620 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857024    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:05:25.869620 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857027    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:05:25.869620 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857030    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:05:25.869989 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.857055    2586 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:05:25.869989 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857088    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:05:25.869989 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857092    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:05:25.869989 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857096    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:05:25.869989 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857099    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:05:25.869989 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857103    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:05:25.869989 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857106    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:05:25.869989 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857108    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:05:25.869989 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857111    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:05:25.869989 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857114    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:05:25.869989 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857116    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:05:25.869989 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857119    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:05:25.869989 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857122    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:05:25.869989 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857124    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:05:25.869989 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857127    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:05:25.869989 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857129    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:05:25.870443 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857132    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:05:25.870443 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857135    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:05:25.870443 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857137    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:05:25.870443 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857140    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:05:25.870443 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857143    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:05:25.870443 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857146    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:05:25.870443 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857148    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:05:25.870443 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857152    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:05:25.870443 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857155    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:05:25.870443 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857157    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:05:25.870443 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857160    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:05:25.870443 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857162    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:05:25.870443 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857165    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:05:25.870443 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857168    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:05:25.870443 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857171    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:05:25.870443 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857175    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:05:25.870443 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857178    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:05:25.870443 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857181    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:05:25.870443 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857184    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:05:25.870928 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857186    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:05:25.870928 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857190    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:05:25.870928 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857194    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:05:25.870928 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857197    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:05:25.870928 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857199    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:05:25.870928 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857202    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:05:25.870928 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857205    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:05:25.870928 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857208    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:05:25.870928 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857221    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:05:25.870928 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857224    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:05:25.870928 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857228    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:05:25.870928 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857232    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:05:25.870928 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857236    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:05:25.870928 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857239    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:05:25.870928 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857241    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:05:25.870928 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857244    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:05:25.870928 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857247    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:05:25.870928 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857250    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:05:25.870928 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857252    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:05:25.871446 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857255    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:05:25.871446 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857260    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:05:25.871446 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857262    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:05:25.871446 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857265    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:05:25.871446 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857267    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:05:25.871446 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857270    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:05:25.871446 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857272    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:05:25.871446 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857275    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:05:25.871446 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857278    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:05:25.871446 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857280    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:05:25.871446 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857283    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:05:25.871446 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857285    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:05:25.871446 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857288    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:05:25.871446 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857291    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:05:25.871446 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857293    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:05:25.871446 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857298    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:05:25.871446 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857300    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:05:25.871446 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857303    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:05:25.871446 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857306    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:05:25.871446 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857308    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:05:25.872093 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857311    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:05:25.872093 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857313    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:05:25.872093 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857316    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:05:25.872093 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857319    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:05:25.872093 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857322    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:05:25.872093 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857324    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:05:25.872093 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857327    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:05:25.872093 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857330    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:05:25.872093 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857332    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:05:25.872093 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857335    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:05:25.872093 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857337    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:05:25.872093 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857340    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:05:25.872093 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857343    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:05:25.872093 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857346    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:05:25.872093 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857349    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:05:25.872093 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857352    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:05:25.872093 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857354    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:05:25.872093 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857357    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:05:25.872093 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857360    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:05:25.872093 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857362    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:05:25.872675 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857365    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:05:25.872675 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857367    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:05:25.872675 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857370    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:05:25.872675 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857372    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:05:25.872675 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857375    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:05:25.872675 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857377    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:05:25.872675 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857380    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:05:25.872675 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857384    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:05:25.872675 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857387    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:05:25.872675 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857390    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:05:25.872675 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857392    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:05:25.872675 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857395    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:05:25.872675 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.857399    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:05:25.872675 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.864635    2586 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 19:05:25.872675 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.864655    2586 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 19:05:25.872675 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.864677    2586 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 19:05:25.872675 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.864685    2586 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 19:05:25.872675 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864769    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:05:25.872675 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864776    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:05:25.873601 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864782    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:05:25.873601 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864786    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:05:25.873601 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864791    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:05:25.873601 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864795    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:05:25.873601 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864800    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:05:25.873601 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864804    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:05:25.873601 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864808    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:05:25.873601 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864813    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:05:25.873601 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864817    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:05:25.873601 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864822    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:05:25.873601 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864825    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:05:25.873601 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864830    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:05:25.873601 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864834    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:05:25.873601 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864838    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:05:25.873601 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864843    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:05:25.873601 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864847    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:05:25.873601 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864852    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:05:25.873601 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864856    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:05:25.873601 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864860    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:05:25.874409 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864864    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:05:25.874409 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864868    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:05:25.874409 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864873    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:05:25.874409 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864877    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:05:25.874409 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864881    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:05:25.874409 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864885    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:05:25.874409 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864889    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:05:25.874409 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864893    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:05:25.874409 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864899    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:05:25.874409 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864904    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:05:25.874409 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864908    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:05:25.874409 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864913    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:05:25.874409 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864916    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:05:25.874409 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864921    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:05:25.874409 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864925    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:05:25.874409 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864929    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:05:25.874409 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864933    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:05:25.874409 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864936    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:05:25.874409 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864941    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:05:25.874409 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864945    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:05:25.875254 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864949    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:05:25.875254 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864953    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:05:25.875254 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864958    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:05:25.875254 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864962    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:05:25.875254 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864966    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:05:25.875254 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864971    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:05:25.875254 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864975    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:05:25.875254 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864979    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:05:25.875254 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864984    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:05:25.875254 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864988    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:05:25.875254 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864993    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:05:25.875254 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.864997    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:05:25.875254 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865002    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:05:25.875254 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865006    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:05:25.875254 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865011    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:05:25.875254 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865015    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:05:25.875254 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865020    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:05:25.875254 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865026    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:05:25.875254 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865033    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:05:25.876002 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865039    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:05:25.876002 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865045    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:05:25.876002 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865052    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:05:25.876002 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865057    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:05:25.876002 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865061    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:05:25.876002 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865066    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:05:25.876002 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865070    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:05:25.876002 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865074    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:05:25.876002 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865078    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:05:25.876002 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865083    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:05:25.876002 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865087    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:05:25.876002 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865092    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:05:25.876002 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865096    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:05:25.876002 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865100    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:05:25.876002 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865105    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:05:25.876002 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865109    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:05:25.876002 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865113    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:05:25.876002 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865117    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:05:25.876002 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865122    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:05:25.876519 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865128    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:05:25.876519 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865134    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:05:25.876519 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865138    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:05:25.876519 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865143    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:05:25.876519 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865148    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:05:25.876519 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865152    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:05:25.876519 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865156    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:05:25.876519 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865160    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:05:25.876519 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865164    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:05:25.876519 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865169    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:05:25.876519 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865173    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:05:25.876519 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865177    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:05:25.876519 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865182    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:05:25.876519 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865186    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:05:25.876519 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865190    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:05:25.876519 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865194    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:05:25.876519 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865199    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:05:25.876519 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865203    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:05:25.876519 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865207    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:05:25.876960 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865227    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:05:25.876960 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865232    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:05:25.876960 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865236    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:05:25.876960 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865241    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:05:25.876960 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865245    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:05:25.876960 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865249    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:05:25.876960 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865253    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:05:25.876960 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865257    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:05:25.876960 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.865297    2586 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:05:25.876960 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865518    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:05:25.876960 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865525    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:05:25.876960 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865530    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:05:25.876960 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865534    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:05:25.876960 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865539    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:05:25.876960 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865543    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:05:25.876960 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865548    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:05:25.877360 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865553    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:05:25.877360 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865557    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:05:25.877360 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865561    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:05:25.877360 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865565    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:05:25.877360 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865570    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:05:25.877360 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865574    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:05:25.877360 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865578    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:05:25.877360 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865582    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:05:25.877360 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865586    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:05:25.877360 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865591    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:05:25.877360 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865595    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:05:25.877360 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865599    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:05:25.877360 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865603    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:05:25.877360 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865607    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:05:25.877360 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865612    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:05:25.877360 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865617    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:05:25.877360 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865621    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:05:25.877360 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865625    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:05:25.877360 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865629    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:05:25.877360 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865634    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:05:25.877819 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865638    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:05:25.877819 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865642    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:05:25.877819 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865646    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:05:25.877819 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865650    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:05:25.877819 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865654    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:05:25.877819 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865659    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:05:25.877819 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865663    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:05:25.877819 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865667    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:05:25.877819 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865671    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:05:25.877819 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865675    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:05:25.877819 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865680    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:05:25.877819 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865684    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:05:25.877819 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865689    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:05:25.877819 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865694    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:05:25.877819 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865700    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:05:25.877819 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865708    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:05:25.877819 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865714    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:05:25.877819 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865718    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:05:25.877819 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865723    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:05:25.878282 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865727    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:05:25.878282 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865731    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:05:25.878282 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865736    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:05:25.878282 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865740    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:05:25.878282 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865744    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:05:25.878282 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865748    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:05:25.878282 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865753    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:05:25.878282 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865757    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:05:25.878282 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865762    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:05:25.878282 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865767    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:05:25.878282 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865772    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:05:25.878282 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865776    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:05:25.878282 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865780    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:05:25.878282 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865784    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:05:25.878282 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865788    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:05:25.878282 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865793    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:05:25.878282 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865797    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:05:25.878282 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865801    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:05:25.878282 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865805    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:05:25.878751 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865810    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:05:25.878751 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865814    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:05:25.878751 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865819    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:05:25.878751 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865823    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:05:25.878751 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865827    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:05:25.878751 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865831    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:05:25.878751 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865836    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:05:25.878751 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865840    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:05:25.878751 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865844    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:05:25.878751 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865849    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:05:25.878751 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865853    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:05:25.878751 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865857    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:05:25.878751 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865862    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:05:25.878751 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865866    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:05:25.878751 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865870    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:05:25.878751 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865874    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:05:25.878751 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865878    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:05:25.878751 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865882    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:05:25.878751 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865886    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:05:25.878751 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865890    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:05:25.879283 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865895    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:05:25.879283 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865899    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:05:25.879283 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865904    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:05:25.879283 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865908    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:05:25.879283 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865912    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:05:25.879283 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865917    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:05:25.879283 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865921    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:05:25.879283 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865925    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:05:25.879283 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865929    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:05:25.879283 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865934    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:05:25.879283 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865938    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:05:25.879283 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865942    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:05:25.879283 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865946    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:05:25.879283 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865951    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:05:25.879283 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865955    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:05:25.879283 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865959    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:05:25.879283 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865963    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:05:25.879283 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865967    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:05:25.879283 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865972    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:05:25.879729 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865978    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:05:25.879729 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.865984    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:05:25.879729 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.866020    2586 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:05:25.879729 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866070    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:05:25.879729 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866077    2586 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:05:25.879729 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866085    2586 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:05:25.879729 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866089    2586 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:05:25.879729 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866093    2586 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:05:25.879729 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866098    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:05:25.879729 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866102    2586 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:05:25.879729 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866107    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:05:25.879729 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866111    2586 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:05:25.879729 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866115    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:05:25.879729 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866119    2586 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:05:25.879729 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866123    2586 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:05:25.880085 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866127    2586 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:05:25.880085 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866134    2586 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:05:25.880085 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866138    2586 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:05:25.880085 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866143    2586 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:05:25.880085 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866147    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:05:25.880085 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866151    2586 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:05:25.880085 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866155    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:05:25.880085 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866161    2586 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:05:25.880085 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866166    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:05:25.880085 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866171    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:05:25.880085 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866175    2586 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:05:25.880085 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866180    2586 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:05:25.880085 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866184    2586 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:05:25.880085 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866189    2586 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:05:25.880085 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866193    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:05:25.880085 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866197    2586 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:05:25.880085 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866205    2586 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:05:25.880085 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866224    2586 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:05:25.880085 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866229    2586 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:05:25.880534 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866234    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:05:25.880534 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866238    2586 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:05:25.880534 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866242    2586 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:05:25.880534 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866247    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:05:25.880534 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866251    2586 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:05:25.880534 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866255    2586 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:05:25.880534 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866259    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:05:25.880534 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866263    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:05:25.880534 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866267    2586 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:05:25.880534 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866272    2586 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:05:25.880534 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866276    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:05:25.880534 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866280    2586 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:05:25.880534 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866284    2586 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:05:25.880534 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866288    2586 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:05:25.880534 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866293    2586 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:05:25.880534 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866298    2586 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:05:25.880534 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866302    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:05:25.880534 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866306    2586 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:05:25.880534 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866310    2586 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:05:25.880534 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866314    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:05:25.881012 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866319    2586 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:05:25.881012 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866323    2586 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:05:25.881012 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866328    2586 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:05:25.881012 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866332    2586 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:05:25.881012 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866336    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:05:25.881012 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866340    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:05:25.881012 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866345    2586 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:05:25.881012 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866349    2586 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:05:25.881012 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866353    2586 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:05:25.881012 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866361    2586 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:05:25.881012 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866365    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:05:25.881012 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866372    2586 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:05:25.881012 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866378    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:05:25.881012 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866383    2586 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:05:25.881012 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866388    2586 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:05:25.881012 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866393    2586 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:05:25.881012 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866399    2586 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:05:25.881012 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866403    2586 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:05:25.881012 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866408    2586 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:05:25.881480 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866412    2586 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:05:25.881480 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866417    2586 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:05:25.881480 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866421    2586 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:05:25.881480 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866426    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:05:25.881480 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866430    2586 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:05:25.881480 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866434    2586 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:05:25.881480 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866438    2586 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:05:25.881480 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866443    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:05:25.881480 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866448    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:05:25.881480 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866453    2586 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:05:25.881480 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866457    2586 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:05:25.881480 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866461    2586 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:05:25.881480 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866465    2586 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:05:25.881480 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866469    2586 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:05:25.881480 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866473    2586 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:05:25.881480 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866478    2586 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:05:25.881480 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866482    2586 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:05:25.881480 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866487    2586 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:05:25.881480 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866492    2586 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:05:25.881976 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866496    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:05:25.881976 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866500    2586 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:05:25.881976 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866505    2586 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:05:25.881976 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866512    2586 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:05:25.881976 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866516    2586 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:05:25.881976 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866520    2586 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:05:25.881976 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866524    2586 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:05:25.881976 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866530    2586 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:05:25.881976 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866534    2586 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:05:25.881976 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866539    2586 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:05:25.881976 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866543    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:05:25.881976 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866547    2586 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:05:25.881976 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866551    2586 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:05:25.881976 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866556    2586 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:05:25.881976 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866560    2586 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:05:25.881976 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866564    2586 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:05:25.881976 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:25.866569    2586 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:05:25.881976 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.866644    2586 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 19:05:25.881976 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.869270    2586 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 19:05:25.882491 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.870207    2586 server.go:1014] "Starting client certificate rotation"
Oct 30 19:05:25.882491 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.870316    2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 19:05:25.882491 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.871080    2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 19:05:25.882491 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.872955    2586 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 19:05:25.882491 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.876966    2586 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 19:05:25.887303 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.887287    2586 log.go:25] "Validated CRI v1 runtime API"
Oct 30 19:05:25.890690 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.890676    2586 log.go:25] "Validated CRI v1 image API"
Oct 30 19:05:25.891486 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.891460    2586 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 19:05:25.894526 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.894507    2586 fs.go:135] Filesystem UUIDs: map[4be381ea-d5f6-46b6-b428-26d1475b7e1b:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 c6e9cc3a-88a0-4416-9e5f-7597206b67a3:/dev/nvme0n1p4]
Oct 30 19:05:25.894589 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.894525    2586 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 19:05:25.899366 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.899259    2586 manager.go:217] Machine: {Timestamp:2026-10-30 19:05:25.898050245 +0000 UTC m=+0.326271645 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100188 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22775fc95798b59cd9a0c3008b4bc0 SystemUUID:ec22775f-c957-98b5-9cd9-a0c3008b4bc0 BootID:ac0e3e80-ac2b-483f-8b1c-61d08d4afbbd Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:fd:f4:f4:53 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:fd:f4:f4:53 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:82:23:10:48:0b:5b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 19:05:25.899366 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.899365    2586 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 19:05:25.899489 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.899473    2586 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 19:05:25.900436 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.900417    2586 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 19:05:25.900561 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.900438    2586 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-200.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 19:05:25.901126 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.901117    2586 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 19:05:25.901160 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.901129    2586 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 19:05:25.901160 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.901151    2586 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 19:05:25.901209 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.901164    2586 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 19:05:25.902032 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.902019    2586 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 19:05:25.905669 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.905657    2586 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 19:05:25.905786 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.905776    2586 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 19:05:25.905827 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.905807    2586 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 19:05:25.905827 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.905817    2586 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 19:05:25.905827 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.905826    2586 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 19:05:25.905901 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.905835    2586 kubelet.go:405] "Adding apiserver pod source"
Oct 30 19:05:25.905901 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.905847    2586 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 19:05:25.906154 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.906141    2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 19:05:25.908487 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.908473    2586 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 19:05:25.909319 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.909303    2586 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 19:05:25.913123 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.913110    2586 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 19:05:25.914025 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.914013    2586 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 19:05:25.914060 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.914043    2586 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 19:05:25.914191 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.914182    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 19:05:25.914233 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.914205    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 19:05:25.914233 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.914223    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 19:05:25.914233 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.914230    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 19:05:25.914311 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.914235    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 19:05:25.914311 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.914241    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 19:05:25.914861 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.914852    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 19:05:25.914892 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.914861    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 19:05:25.914892 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.914868    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 19:05:25.914892 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.914874    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 19:05:25.914892 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.914887    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 19:05:25.915010 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.914895    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 19:05:25.916105 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.916096    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 19:05:25.916105 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.916105    2586 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 19:05:25.921496 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.921480    2586 server.go:1263] "Started kubelet"
Oct 30 19:05:25.921623 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.921563    2586 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 19:05:25.921623 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.921568    2586 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 19:05:25.921723 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.921628    2586 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 19:05:25.922183 ip-10-0-137-200 systemd[1]: Started Kubernetes Kubelet.
Oct 30 19:05:25.922806 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.922789    2586 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 19:05:25.923233 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.923125    2586 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-200.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 19:05:25.924175 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.924120    2586 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 19:05:25.926260 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.926237    2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7rn4x"
Oct 30 19:05:25.926748 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:25.926731    2586 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 19:05:25.928720 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.928701    2586 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 19:05:25.928720 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.928715    2586 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 19:05:25.928854 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:25.928783    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-200.us-east-2.compute.internal\" not found"
Oct 30 19:05:25.929097 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:25.929068    2586 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 19:05:25.929097 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:25.929076    2586 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-200.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 19:05:25.929316 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.929298    2586 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 19:05:25.929316 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.929301    2586 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 19:05:25.929455 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.929327    2586 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 19:05:25.929455 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.929415    2586 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 19:05:25.929455 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.929425    2586 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 19:05:25.930072 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.930056    2586 factory.go:55] Registering systemd factory
Oct 30 19:05:25.930072 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.930074    2586 factory.go:223] Registration of the systemd container factory successfully
Oct 30 19:05:25.930194 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:25.929174    2586 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-200.us-east-2.compute.internal.18e36416d94e9566  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-200.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-200.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 19:05:25.921453414 +0000 UTC m=+0.349674816,LastTimestamp:2026-10-30 19:05:25.921453414 +0000 UTC m=+0.349674816,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-200.us-east-2.compute.internal,}"
Oct 30 19:05:25.930302 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.930287    2586 factory.go:153] Registering CRI-O factory
Oct 30 19:05:25.930352 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.930307    2586 factory.go:223] Registration of the crio container factory successfully
Oct 30 19:05:25.930399 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.930370    2586 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 19:05:25.930399 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.930388    2586 factory.go:103] Registering Raw factory
Oct 30 19:05:25.930496 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.930402    2586 manager.go:1199] Started watching for new ooms in manager
Oct 30 19:05:25.931354 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.931338    2586 manager.go:319] Starting recovery of all containers
Oct 30 19:05:25.938856 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:25.938822    2586 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-200.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 19:05:25.940030 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.940008    2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7rn4x"
Oct 30 19:05:25.942615 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.942597    2586 manager.go:324] Recovery completed
Oct 30 19:05:25.943328 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:25.943293    2586 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 19:05:25.945733 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:25.944955    2586 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Oct 30 19:05:25.948045 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.947944    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:05:25.949798 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.949779    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:05:25.949878 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.949816    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:05:25.949878 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.949830    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:05:25.950430 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.950416    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:05:25.950484 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.950417    2586 cpu_manager.go:226] "Starting" policy="none"
Oct 30 19:05:25.950484 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.950472    2586 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 19:05:25.950556 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.950491    2586 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 19:05:25.951068 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.951052    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:05:25.951122 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.951083    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:05:25.951122 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.951098    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:05:25.952580 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.952567    2586 policy_none.go:50] "Start"
Oct 30 19:05:25.953012 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.952988    2586 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 19:05:25.953097 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.953024    2586 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 19:05:25.953163 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:25.953044    2586 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-200.us-east-2.compute.internal.18e36416daff1860  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-200.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-200.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-200.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 19:05:25.949798496 +0000 UTC m=+0.378019903,LastTimestamp:2026-10-30 19:05:25.949798496 +0000 UTC m=+0.378019903,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-200.us-east-2.compute.internal,}"
Oct 30 19:05:25.955042 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.955028    2586 policy_none.go:44] "Start"
Oct 30 19:05:25.980859 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.980831    2586 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 19:05:26.010115 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.985472    2586 manager.go:341] "Starting Device Plugin manager"
Oct 30 19:05:26.010115 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:25.986024    2586 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 19:05:26.010115 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.986040    2586 server.go:87] "Starting device plugin registration server"
Oct 30 19:05:26.010115 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.986254    2586 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 19:05:26.010115 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.986263    2586 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 19:05:26.010115 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.986368    2586 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 19:05:26.010115 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.986443    2586 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 19:05:26.010115 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.986452    2586 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 19:05:26.010115 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:25.986869    2586 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 19:05:26.010115 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:25.986916    2586 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-200.us-east-2.compute.internal\" not found"
Oct 30 19:05:26.010115 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.995149    2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 19:05:26.010115 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.996258    2586 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 19:05:26.010115 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.996275    2586 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 19:05:26.010115 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:25.996299    2586 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 19:05:26.010115 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:25.996334    2586 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 19:05:26.010115 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.000802    2586 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:05:26.086980 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.086957    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:05:26.087758 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.087744    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:05:26.087827 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.087770    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:05:26.087827 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.087785    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:05:26.087827 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.087814    2586 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:05:26.097180 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.097153    2586 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal"]
Oct 30 19:05:26.097238 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.097222    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:05:26.097955 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.097906    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:05:26.097955 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.097931    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:05:26.097955 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.097946    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:05:26.098205 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.098190    2586 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:05:26.098252 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:26.098222    2586 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-200.us-east-2.compute.internal\": node \"ip-10-0-137-200.us-east-2.compute.internal\" not found"
Oct 30 19:05:26.100111 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.100100    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:05:26.100266 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.100251    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:05:26.100338 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.100287    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:05:26.102567 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.102550    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:05:26.102669 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.102573    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:05:26.102669 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.102583    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:05:26.102669 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.102592    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:05:26.102669 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.102616    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:05:26.102669 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.102630    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:05:26.104734 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.104717    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:05:26.104813 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.104766    2586 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:05:26.105553 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.105538    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:05:26.105633 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.105563    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:05:26.105633 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.105574    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:05:26.114570 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:26.114551    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-200.us-east-2.compute.internal\" not found"
Oct 30 19:05:26.137625 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:26.137609    2586 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-200.us-east-2.compute.internal\" not found" node="ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:05:26.140508 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:26.140491    2586 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-200.us-east-2.compute.internal\" not found" node="ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:05:26.215165 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:26.215141    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-200.us-east-2.compute.internal\" not found"
Oct 30 19:05:26.230419 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.230395    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7fe3dcaf32fbbd51ccedffe35a8755ab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal\" (UID: \"7fe3dcaf32fbbd51ccedffe35a8755ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:05:26.230499 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.230428    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7fe3dcaf32fbbd51ccedffe35a8755ab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal\" (UID: \"7fe3dcaf32fbbd51ccedffe35a8755ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:05:26.230499 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.230455    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7e14d55fcbb7a4643b4d48f7935b8114-config\") pod \"kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal\" (UID: \"7e14d55fcbb7a4643b4d48f7935b8114\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:05:26.315627 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:26.315602    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-200.us-east-2.compute.internal\" not found"
Oct 30 19:05:26.331100 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.331084    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7fe3dcaf32fbbd51ccedffe35a8755ab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal\" (UID: \"7fe3dcaf32fbbd51ccedffe35a8755ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:05:26.331194 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.331107    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7fe3dcaf32fbbd51ccedffe35a8755ab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal\" (UID: \"7fe3dcaf32fbbd51ccedffe35a8755ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:05:26.331194 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.331122    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7fe3dcaf32fbbd51ccedffe35a8755ab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal\" (UID: \"7fe3dcaf32fbbd51ccedffe35a8755ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:05:26.331194 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.331125    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7e14d55fcbb7a4643b4d48f7935b8114-config\") pod \"kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal\" (UID: \"7e14d55fcbb7a4643b4d48f7935b8114\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:05:26.331194 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.331169    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7e14d55fcbb7a4643b4d48f7935b8114-config\") pod \"kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal\" (UID: \"7e14d55fcbb7a4643b4d48f7935b8114\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:05:26.331346 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.331190    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7fe3dcaf32fbbd51ccedffe35a8755ab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal\" (UID: \"7fe3dcaf32fbbd51ccedffe35a8755ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:05:26.416512 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:26.416483    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-200.us-east-2.compute.internal\" not found"
Oct 30 19:05:26.440924 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.440906    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:05:26.444083 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.444064    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:05:26.517440 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:26.517273    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-200.us-east-2.compute.internal\" not found"
Oct 30 19:05:26.617792 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:26.617764    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-200.us-east-2.compute.internal\" not found"
Oct 30 19:05:26.718298 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:26.718249    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-200.us-east-2.compute.internal\" not found"
Oct 30 19:05:26.818851 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:26.818825    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-200.us-east-2.compute.internal\" not found"
Oct 30 19:05:26.870369 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.870345    2586 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 19:05:26.870769 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:26.870476    2586 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 19:05:28.623774 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:28.623751    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-200.us-east-2.compute.internal\" not found"
Oct 30 19:05:28.650139 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:28.650105    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 19:00:27 +0000 UTC" deadline="2028-05-25 04:39:56.45291024 +0000 UTC"
Oct 30 19:05:28.650139 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:28.650136    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13737h34m27.802777947s"
Oct 30 19:05:28.724148 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:28.724089    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-200.us-east-2.compute.internal\" not found"
Oct 30 19:05:28.753043 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:28.753007    2586 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:05:28.786568 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:28.786538    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fe3dcaf32fbbd51ccedffe35a8755ab.slice/crio-c5816817ad4d94d47e364d6a051169d0f69b695fc2f17e86ec0af6250b70843b WatchSource:0}: Error finding container c5816817ad4d94d47e364d6a051169d0f69b695fc2f17e86ec0af6250b70843b: Status 404 returned error can't find the container with id c5816817ad4d94d47e364d6a051169d0f69b695fc2f17e86ec0af6250b70843b
Oct 30 19:05:28.786742 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:28.786724    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e14d55fcbb7a4643b4d48f7935b8114.slice/crio-1b6e265adf5cb80c3712e09bc95d489b247261151cb0892a6e6f2e85afb2e715 WatchSource:0}: Error finding container 1b6e265adf5cb80c3712e09bc95d489b247261151cb0892a6e6f2e85afb2e715: Status 404 returned error can't find the container with id 1b6e265adf5cb80c3712e09bc95d489b247261151cb0892a6e6f2e85afb2e715
Oct 30 19:05:28.791950 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:28.791936    2586 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:05:28.824623 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:28.824603    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-200.us-east-2.compute.internal\" not found"
Oct 30 19:05:28.925113 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:28.925088    2586 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-200.us-east-2.compute.internal\" not found"
Oct 30 19:05:28.948818 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:28.948802    2586 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:05:29.033002 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.032942    2586 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:05:29.041964 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.041945    2586 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:05:29.042905 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.042893    2586 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:05:29.050510 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.050497    2586 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:05:29.054221 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.054208    2586 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:05:29.536972 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.536942    2586 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:05:29.611340 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.611307    2586 apiserver.go:52] "Watching apiserver"
Oct 30 19:05:29.622869 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.622842    2586 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 19:05:29.623273 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.623244    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-ln5ql","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn","openshift-dns/node-resolver-8jrs8","openshift-image-registry/node-ca-2z2fd","openshift-multus/multus-lq697","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal","openshift-multus/multus-additional-cni-plugins-d2ntn","openshift-network-diagnostics/network-check-target-z2dbh","openshift-network-operator/iptables-alerter-4q28p","openshift-ovn-kubernetes/ovnkube-node-2gqlw","kube-system/global-pull-secret-syncer-7q5vl","openshift-cluster-node-tuning-operator/tuned-b7bcf","openshift-multus/network-metrics-daemon-xmkt2","kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal"]
Oct 30 19:05:29.625899 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.625868    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.628339 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.628319    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:29.628433 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:29.628397    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmkt2" podUID="ed4e7b61-d816-49fc-b4d9-36236a716227"
Oct 30 19:05:29.632117 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.630555    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8jrs8"
Oct 30 19:05:29.632117 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.630686    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 19:05:29.632117 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.630770    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jks97\""
Oct 30 19:05:29.632117 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.630875    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 19:05:29.632117 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.630957    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 19:05:29.632117 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.630976    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 19:05:29.632117 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.632111    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 19:05:29.632591 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.632565    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2z2fd"
Oct 30 19:05:29.633689 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.633668    2586 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 19:05:29.634972 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.634956    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.636151 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.636129    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 19:05:29.637446 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.637429    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:29.637542 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:29.637483    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2dbh" podUID="c1cc12f1-be52-4a9f-a8a2-53fe3c493431"
Oct 30 19:05:29.639734 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.639714    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4q28p"
Oct 30 19:05:29.641954 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.641935    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ln5ql"
Oct 30 19:05:29.643164 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643144    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7c1242a-b12d-49b2-8052-781394f01418-cni-binary-copy\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.643164 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643207    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/838f8f5e-ac73-4280-8ed3-173a51c42b9e-serviceca\") pod \"node-ca-2z2fd\" (UID: \"838f8f5e-ac73-4280-8ed3-173a51c42b9e\") " pod="openshift-image-registry/node-ca-2z2fd"
Oct 30 19:05:29.643164 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643233    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-var-lib-kubelet\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.643434 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643256    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-daemon-config\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.643434 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643278    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-cni-dir\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.643434 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643300    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-run-netns\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.643434 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643355    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7c1242a-b12d-49b2-8052-781394f01418-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.643434 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643385    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-run-k8s-cni-cncf-io\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.643434 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643408    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/838f8f5e-ac73-4280-8ed3-173a51c42b9e-host\") pod \"node-ca-2z2fd\" (UID: \"838f8f5e-ac73-4280-8ed3-173a51c42b9e\") " pod="openshift-image-registry/node-ca-2z2fd"
Oct 30 19:05:29.643434 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643429    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpgcn\" (UniqueName: \"kubernetes.io/projected/838f8f5e-ac73-4280-8ed3-173a51c42b9e-kube-api-access-zpgcn\") pod \"node-ca-2z2fd\" (UID: \"838f8f5e-ac73-4280-8ed3-173a51c42b9e\") " pod="openshift-image-registry/node-ca-2z2fd"
Oct 30 19:05:29.643661 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643455    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-run-multus-certs\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.643661 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643498    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94709da2-704a-421c-aa2f-1844db5bec96-tmp-dir\") pod \"node-resolver-8jrs8\" (UID: \"94709da2-704a-421c-aa2f-1844db5bec96\") " pod="openshift-dns/node-resolver-8jrs8"
Oct 30 19:05:29.643661 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643532    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7c1242a-b12d-49b2-8052-781394f01418-system-cni-dir\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.643661 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643556    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7c1242a-b12d-49b2-8052-781394f01418-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.643661 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643592    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-socket-dir-parent\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.643661 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643621    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-var-lib-cni-multus\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.643661 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643654    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-conf-dir\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.643973 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643717    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-etc-kubernetes\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.643973 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643786    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7c1242a-b12d-49b2-8052-781394f01418-cnibin\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.643973 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643812    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4hr8\" (UniqueName: \"kubernetes.io/projected/ed4e7b61-d816-49fc-b4d9-36236a716227-kube-api-access-b4hr8\") pod \"network-metrics-daemon-xmkt2\" (UID: \"ed4e7b61-d816-49fc-b4d9-36236a716227\") " pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:29.643973 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643854    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-system-cni-dir\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.643973 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643887    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65lrg\" (UniqueName: \"kubernetes.io/projected/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-kube-api-access-65lrg\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.643973 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643932    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-var-lib-cni-bin\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.644239 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.643983    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/94709da2-704a-421c-aa2f-1844db5bec96-hosts-file\") pod \"node-resolver-8jrs8\" (UID: \"94709da2-704a-421c-aa2f-1844db5bec96\") " pod="openshift-dns/node-resolver-8jrs8"
Oct 30 19:05:29.644239 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.644005    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvx2d\" (UniqueName: \"kubernetes.io/projected/b7c1242a-b12d-49b2-8052-781394f01418-kube-api-access-qvx2d\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.644239 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.644020    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs\") pod \"network-metrics-daemon-xmkt2\" (UID: \"ed4e7b61-d816-49fc-b4d9-36236a716227\") " pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:29.644239 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.644035    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-cnibin\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.644239 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.644048    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-cni-binary-copy\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.644239 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.644071    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-hostroot\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.644239 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.644094    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7c1242a-b12d-49b2-8052-781394f01418-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.644239 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.644117    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hf6x\" (UniqueName: \"kubernetes.io/projected/94709da2-704a-421c-aa2f-1844db5bec96-kube-api-access-6hf6x\") pod \"node-resolver-8jrs8\" (UID: \"94709da2-704a-421c-aa2f-1844db5bec96\") " pod="openshift-dns/node-resolver-8jrs8"
Oct 30 19:05:29.644612 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.644421    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.646635 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.646616    2586 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 19:05:29.646921 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.646899    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.649741 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.649723    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:29.649833 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:29.649780    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q5vl" podUID="ba82c577-b89d-44f7-87e4-2d00d1e2a5de"
Oct 30 19:05:29.653187 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.653158    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.665141 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.665119    2586 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kgkt5"
Oct 30 19:05:29.677511 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.677491    2586 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kgkt5"
Oct 30 19:05:29.705655 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.705617    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal" event={"ID":"7fe3dcaf32fbbd51ccedffe35a8755ab","Type":"ContainerStarted","Data":"c5816817ad4d94d47e364d6a051169d0f69b695fc2f17e86ec0af6250b70843b"}
Oct 30 19:05:29.706768 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.706742    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal" event={"ID":"7e14d55fcbb7a4643b4d48f7935b8114","Type":"ContainerStarted","Data":"1b6e265adf5cb80c3712e09bc95d489b247261151cb0892a6e6f2e85afb2e715"}
Oct 30 19:05:29.730661 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.730623    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 19:05:29.730661 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.730629    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 19:05:29.730811 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.730715    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 19:05:29.730811 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.730772    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:05:29.730909 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.730839    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 19:05:29.730909 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.730902    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hsp67\""
Oct 30 19:05:29.731010 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.730949    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 19:05:29.731411 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.731392    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 19:05:29.731822 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.731806    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 19:05:29.732165 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.732077    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 19:05:29.732165 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.732111    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 19:05:29.732165 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.732128    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 19:05:29.732165 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.732139    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 19:05:29.732359 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.732248    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xlxzb\""
Oct 30 19:05:29.732407 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.732384    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 19:05:29.732456 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.732439    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-m74bb\""
Oct 30 19:05:29.732500 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.732459    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 19:05:29.733109 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.733034    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 19:05:29.733109 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.733040    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:05:29.733109 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.733080    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gkzrx\""
Oct 30 19:05:29.733275 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.733091    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 19:05:29.733275 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.733094    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-4z6gp\""
Oct 30 19:05:29.733373 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.733354    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 19:05:29.734130 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.734108    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 19:05:29.734252 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.734152    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xd9sf\""
Oct 30 19:05:29.734252 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.734191    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 19:05:29.734252 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.734216    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-w2h58\""
Oct 30 19:05:29.734252 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.734222    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ngr9b\""
Oct 30 19:05:29.734428 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.734399    2586 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 19:05:29.734840 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.734825    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 19:05:29.744991 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.744968    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-cni-binary-copy\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.745097 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745004    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-hostroot\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.745097 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745029    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-registration-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.745097 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745052    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-systemd-units\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.745097 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745076    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7c1242a-b12d-49b2-8052-781394f01418-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.745269 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745101    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-sys-fs\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.745269 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745123    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-kubelet\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.745269 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745146    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-run-ovn-kubernetes\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.745269 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745184    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-ovn-node-metrics-cert\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.745269 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745207    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-ovnkube-script-lib\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.745269 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745231    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd24c416-9f3c-4955-a536-08f20ccdbbbb-host-slash\") pod \"iptables-alerter-4q28p\" (UID: \"fd24c416-9f3c-4955-a536-08f20ccdbbbb\") " pod="openshift-network-operator/iptables-alerter-4q28p"
Oct 30 19:05:29.745269 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745255    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7c1242a-b12d-49b2-8052-781394f01418-cni-binary-copy\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.745599 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745277    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/838f8f5e-ac73-4280-8ed3-173a51c42b9e-serviceca\") pod \"node-ca-2z2fd\" (UID: \"838f8f5e-ac73-4280-8ed3-173a51c42b9e\") " pod="openshift-image-registry/node-ca-2z2fd"
Oct 30 19:05:29.745599 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745299    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-var-lib-kubelet\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.745599 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745321    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-sysctl-d\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.745599 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745335    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-tuned\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.745599 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745350    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-run-netns\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.745599 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745382    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nlrh\" (UniqueName: \"kubernetes.io/projected/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-kube-api-access-9nlrh\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.745599 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745406    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49bwq\" (UniqueName: \"kubernetes.io/projected/fd24c416-9f3c-4955-a536-08f20ccdbbbb-kube-api-access-49bwq\") pod \"iptables-alerter-4q28p\" (UID: \"fd24c416-9f3c-4955-a536-08f20ccdbbbb\") " pod="openshift-network-operator/iptables-alerter-4q28p"
Oct 30 19:05:29.745599 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745426    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-lib-modules\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.745599 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745447    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-host\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.745599 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745483    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7c1242a-b12d-49b2-8052-781394f01418-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.745599 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745498    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-run-systemd\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.745599 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745521    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-cni-bin\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.745599 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745537    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.745599 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745559    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fd24c416-9f3c-4955-a536-08f20ccdbbbb-iptables-alerter-script\") pod \"iptables-alerter-4q28p\" (UID: \"fd24c416-9f3c-4955-a536-08f20ccdbbbb\") " pod="openshift-network-operator/iptables-alerter-4q28p"
Oct 30 19:05:29.745599 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745580    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-dbus\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:29.745599 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745601    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/838f8f5e-ac73-4280-8ed3-173a51c42b9e-host\") pod \"node-ca-2z2fd\" (UID: \"838f8f5e-ac73-4280-8ed3-173a51c42b9e\") " pod="openshift-image-registry/node-ca-2z2fd"
Oct 30 19:05:29.746296 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745622    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-run-multus-certs\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.746296 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745641    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-socket-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.746296 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745660    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-etc-selinux\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.746296 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745674    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-slash\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.746296 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745688    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-run-ovn\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.746296 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745714    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-node-log\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.746296 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745731    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-modprobe-d\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.746296 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745747    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7c1242a-b12d-49b2-8052-781394f01418-system-cni-dir\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.746296 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745765    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7c1242a-b12d-49b2-8052-781394f01418-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.746296 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745781    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-conf-dir\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.746296 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745795    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65lrg\" (UniqueName: \"kubernetes.io/projected/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-kube-api-access-65lrg\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.746296 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745812    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:29.746296 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.745852    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-sysconfig\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.746296 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746032    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-hostroot\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.746296 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746075    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-sys\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.746296 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746105    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-var-lib-kubelet\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.746296 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746134    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7c1242a-b12d-49b2-8052-781394f01418-cnibin\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.747061 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746161    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-system-cni-dir\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.747061 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746207    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-cni-binary-copy\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.747061 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746205    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pq2q\" (UniqueName: \"kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q\") pod \"network-check-target-z2dbh\" (UID: \"c1cc12f1-be52-4a9f-a8a2-53fe3c493431\") " pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:29.747061 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746242    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.747061 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746258    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-run-openvswitch\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.747061 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746273    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-cni-netd\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.747061 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746288    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-kubelet-config\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:29.747061 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746294    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7c1242a-b12d-49b2-8052-781394f01418-system-cni-dir\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.747061 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746306    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-kubernetes\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.747061 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746338    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs\") pod \"network-metrics-daemon-xmkt2\" (UID: \"ed4e7b61-d816-49fc-b4d9-36236a716227\") " pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:29.747061 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746363    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-run\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.747061 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746380    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-tmp\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.747061 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746397    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdsdn\" (UniqueName: \"kubernetes.io/projected/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-kube-api-access-qdsdn\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.747061 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746418    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hf6x\" (UniqueName: \"kubernetes.io/projected/94709da2-704a-421c-aa2f-1844db5bec96-kube-api-access-6hf6x\") pod \"node-resolver-8jrs8\" (UID: \"94709da2-704a-421c-aa2f-1844db5bec96\") " pod="openshift-dns/node-resolver-8jrs8"
Oct 30 19:05:29.747061 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746438    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8d23c4bb-0012-4e09-bc59-c6689f9ea394-agent-certs\") pod \"konnectivity-agent-ln5ql\" (UID: \"8d23c4bb-0012-4e09-bc59-c6689f9ea394\") " pod="kube-system/konnectivity-agent-ln5ql"
Oct 30 19:05:29.747061 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746452    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-run-netns\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.747061 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746478    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-daemon-config\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.748799 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746502    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-etc-openvswitch\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.748799 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746521    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-log-socket\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.748799 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746537    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-env-overrides\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.748799 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746553    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxvt7\" (UniqueName: \"kubernetes.io/projected/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-kube-api-access-qxvt7\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.748799 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746570    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-sysctl-conf\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.748799 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746588    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-systemd\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.748799 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746605    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-cni-dir\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.748799 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746621    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-var-lib-openvswitch\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.748799 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746637    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-ovnkube-config\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.748799 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746664    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-run-k8s-cni-cncf-io\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.748799 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746678    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/838f8f5e-ac73-4280-8ed3-173a51c42b9e-serviceca\") pod \"node-ca-2z2fd\" (UID: \"838f8f5e-ac73-4280-8ed3-173a51c42b9e\") " pod="openshift-image-registry/node-ca-2z2fd"
Oct 30 19:05:29.748799 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746691    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8d23c4bb-0012-4e09-bc59-c6689f9ea394-konnectivity-ca\") pod \"konnectivity-agent-ln5ql\" (UID: \"8d23c4bb-0012-4e09-bc59-c6689f9ea394\") " pod="kube-system/konnectivity-agent-ln5ql"
Oct 30 19:05:29.748799 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746728    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpgcn\" (UniqueName: \"kubernetes.io/projected/838f8f5e-ac73-4280-8ed3-173a51c42b9e-kube-api-access-zpgcn\") pod \"node-ca-2z2fd\" (UID: \"838f8f5e-ac73-4280-8ed3-173a51c42b9e\") " pod="openshift-image-registry/node-ca-2z2fd"
Oct 30 19:05:29.748799 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746771    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7c1242a-b12d-49b2-8052-781394f01418-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.748799 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746811    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-conf-dir\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.748799 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746823    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-run-netns\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.748799 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746833    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-var-lib-kubelet\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.749421 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746904    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-cni-dir\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.749421 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:29.746910    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:05:29.749421 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746939    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-run-k8s-cni-cncf-io\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.749421 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:29.746967    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs podName:ed4e7b61-d816-49fc-b4d9-36236a716227 nodeName:}" failed. No retries permitted until 2026-10-30 19:05:30.246948878 +0000 UTC m=+2.971213593 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs") pod "network-metrics-daemon-xmkt2" (UID: "ed4e7b61-d816-49fc-b4d9-36236a716227") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:05:29.749421 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.746988    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94709da2-704a-421c-aa2f-1844db5bec96-tmp-dir\") pod \"node-resolver-8jrs8\" (UID: \"94709da2-704a-421c-aa2f-1844db5bec96\") " pod="openshift-dns/node-resolver-8jrs8"
Oct 30 19:05:29.749421 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747035    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-socket-dir-parent\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.749421 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747064    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-var-lib-cni-multus\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.749421 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747088    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-etc-kubernetes\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.749421 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747102    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-system-cni-dir\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.749421 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747119    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-device-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.749421 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747148    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4hr8\" (UniqueName: \"kubernetes.io/projected/ed4e7b61-d816-49fc-b4d9-36236a716227-kube-api-access-b4hr8\") pod \"network-metrics-daemon-xmkt2\" (UID: \"ed4e7b61-d816-49fc-b4d9-36236a716227\") " pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:29.749421 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747040    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7c1242a-b12d-49b2-8052-781394f01418-cnibin\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.749421 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747252    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7c1242a-b12d-49b2-8052-781394f01418-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.749421 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747276    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-var-lib-cni-bin\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.749421 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747305    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/94709da2-704a-421c-aa2f-1844db5bec96-hosts-file\") pod \"node-resolver-8jrs8\" (UID: \"94709da2-704a-421c-aa2f-1844db5bec96\") " pod="openshift-dns/node-resolver-8jrs8"
Oct 30 19:05:29.749421 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747332    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvx2d\" (UniqueName: \"kubernetes.io/projected/b7c1242a-b12d-49b2-8052-781394f01418-kube-api-access-qvx2d\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.749421 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747357    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-cnibin\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.750147 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747433    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-cnibin\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.750147 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747468    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-run-multus-certs\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.750147 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747496    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-daemon-config\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.750147 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747549    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7c1242a-b12d-49b2-8052-781394f01418-cni-binary-copy\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.750147 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747581    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/838f8f5e-ac73-4280-8ed3-173a51c42b9e-host\") pod \"node-ca-2z2fd\" (UID: \"838f8f5e-ac73-4280-8ed3-173a51c42b9e\") " pod="openshift-image-registry/node-ca-2z2fd"
Oct 30 19:05:29.750147 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747583    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-socket-dir-parent\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.750147 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747625    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-etc-kubernetes\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.750147 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747623    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-var-lib-cni-multus\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.750147 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747684    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-var-lib-cni-bin\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.750147 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747939    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94709da2-704a-421c-aa2f-1844db5bec96-tmp-dir\") pod \"node-resolver-8jrs8\" (UID: \"94709da2-704a-421c-aa2f-1844db5bec96\") " pod="openshift-dns/node-resolver-8jrs8"
Oct 30 19:05:29.750147 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747953    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/94709da2-704a-421c-aa2f-1844db5bec96-hosts-file\") pod \"node-resolver-8jrs8\" (UID: \"94709da2-704a-421c-aa2f-1844db5bec96\") " pod="openshift-dns/node-resolver-8jrs8"
Oct 30 19:05:29.750147 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.747997    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7c1242a-b12d-49b2-8052-781394f01418-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.759876 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.759849    2586 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 19:05:29.764331 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.764304    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65lrg\" (UniqueName: \"kubernetes.io/projected/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-kube-api-access-65lrg\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.764451 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.764434    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvx2d\" (UniqueName: \"kubernetes.io/projected/b7c1242a-b12d-49b2-8052-781394f01418-kube-api-access-qvx2d\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.767580 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.767554    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hf6x\" (UniqueName: \"kubernetes.io/projected/94709da2-704a-421c-aa2f-1844db5bec96-kube-api-access-6hf6x\") pod \"node-resolver-8jrs8\" (UID: \"94709da2-704a-421c-aa2f-1844db5bec96\") " pod="openshift-dns/node-resolver-8jrs8"
Oct 30 19:05:29.767674 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.767584    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4hr8\" (UniqueName: \"kubernetes.io/projected/ed4e7b61-d816-49fc-b4d9-36236a716227-kube-api-access-b4hr8\") pod \"network-metrics-daemon-xmkt2\" (UID: \"ed4e7b61-d816-49fc-b4d9-36236a716227\") " pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:29.768297 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.768272    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpgcn\" (UniqueName: \"kubernetes.io/projected/838f8f5e-ac73-4280-8ed3-173a51c42b9e-kube-api-access-zpgcn\") pod \"node-ca-2z2fd\" (UID: \"838f8f5e-ac73-4280-8ed3-173a51c42b9e\") " pod="openshift-image-registry/node-ca-2z2fd"
Oct 30 19:05:29.848004 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.847967    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8d23c4bb-0012-4e09-bc59-c6689f9ea394-konnectivity-ca\") pod \"konnectivity-agent-ln5ql\" (UID: \"8d23c4bb-0012-4e09-bc59-c6689f9ea394\") " pod="kube-system/konnectivity-agent-ln5ql"
Oct 30 19:05:29.848004 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848007    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-device-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.848245 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848031    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-registration-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.848245 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848051    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-systemd-units\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.848245 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848073    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-sys-fs\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.848245 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848093    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-kubelet\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.848245 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848112    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-run-ovn-kubernetes\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.848245 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848133    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-ovn-node-metrics-cert\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.848245 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848154    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-ovnkube-script-lib\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.848245 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848190    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd24c416-9f3c-4955-a536-08f20ccdbbbb-host-slash\") pod \"iptables-alerter-4q28p\" (UID: \"fd24c416-9f3c-4955-a536-08f20ccdbbbb\") " pod="openshift-network-operator/iptables-alerter-4q28p"
Oct 30 19:05:29.848245 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848215    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-sysctl-d\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.848245 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848239    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-tuned\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.848715 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848263    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nlrh\" (UniqueName: \"kubernetes.io/projected/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-kube-api-access-9nlrh\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.848715 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848288    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49bwq\" (UniqueName: \"kubernetes.io/projected/fd24c416-9f3c-4955-a536-08f20ccdbbbb-kube-api-access-49bwq\") pod \"iptables-alerter-4q28p\" (UID: \"fd24c416-9f3c-4955-a536-08f20ccdbbbb\") " pod="openshift-network-operator/iptables-alerter-4q28p"
Oct 30 19:05:29.848715 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848310    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-lib-modules\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.848715 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848333    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-host\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.848715 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848355    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-run-systemd\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.848715 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848379    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-cni-bin\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.848715 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848405    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.848715 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848432    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fd24c416-9f3c-4955-a536-08f20ccdbbbb-iptables-alerter-script\") pod \"iptables-alerter-4q28p\" (UID: \"fd24c416-9f3c-4955-a536-08f20ccdbbbb\") " pod="openshift-network-operator/iptables-alerter-4q28p"
Oct 30 19:05:29.848715 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848455    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-dbus\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:29.848715 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848482    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-socket-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.848715 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848505    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-etc-selinux\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.848715 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848529    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-slash\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.848715 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848552    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-run-ovn\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.848715 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848573    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-node-log\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.848715 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848597    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-modprobe-d\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.848715 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848627    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:29.848715 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848652    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-sysconfig\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.849467 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848673    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-sys\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.849467 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848696    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-var-lib-kubelet\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.849467 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848722    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pq2q\" (UniqueName: \"kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q\") pod \"network-check-target-z2dbh\" (UID: \"c1cc12f1-be52-4a9f-a8a2-53fe3c493431\") " pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:29.849467 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848746    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.849467 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848778    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-run-openvswitch\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.849467 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848807    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-cni-netd\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.849467 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848832    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-kubelet-config\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:29.849467 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848858    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-kubernetes\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.849467 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848894    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-run\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.849467 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848921    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-tmp\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.849467 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848945    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdsdn\" (UniqueName: \"kubernetes.io/projected/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-kube-api-access-qdsdn\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.849467 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848970    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8d23c4bb-0012-4e09-bc59-c6689f9ea394-agent-certs\") pod \"konnectivity-agent-ln5ql\" (UID: \"8d23c4bb-0012-4e09-bc59-c6689f9ea394\") " pod="kube-system/konnectivity-agent-ln5ql"
Oct 30 19:05:29.849467 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.848995    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-run-netns\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.849467 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849020    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-etc-openvswitch\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.849467 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849043    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-log-socket\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.849467 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849068    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-env-overrides\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.849467 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849094    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxvt7\" (UniqueName: \"kubernetes.io/projected/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-kube-api-access-qxvt7\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.850243 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849120    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-sysctl-conf\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.850243 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849143    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-systemd\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.850243 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849191    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-var-lib-openvswitch\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.850243 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849261    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-slash\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.850243 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849292    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-run-systemd\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.850243 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849303    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-kubernetes\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.850243 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849292    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-etc-openvswitch\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.850243 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849343    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-sys-fs\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.850243 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849347    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-run-ovn-kubernetes\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.850243 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849359    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-cni-bin\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.850243 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849393    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-kubelet\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.850243 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849394    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-sysctl-conf\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.850243 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849406    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.850243 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849412    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-run\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.850243 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849421    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-device-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.850243 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849428    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-var-lib-openvswitch\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.850243 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849440    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-ovnkube-config\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.851020 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849309    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-sys\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.851020 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849475    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-kubelet-config\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:29.851020 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849481    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-node-log\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.851020 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849495    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-registration-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.851020 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849512    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-systemd-units\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.851020 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:29.849515    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:05:29.851020 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849556    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-cni-netd\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.851020 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849557    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.851020 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:29.849568    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret podName:ba82c577-b89d-44f7-87e4-2d00d1e2a5de nodeName:}" failed. No retries permitted until 2026-10-30 19:05:30.349551056 +0000 UTC m=+3.073815765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret") pod "global-pull-secret-syncer-7q5vl" (UID: "ba82c577-b89d-44f7-87e4-2d00d1e2a5de") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:05:29.851020 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849593    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-systemd\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.851020 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849595    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-run-openvswitch\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.851020 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849518    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-run-ovn\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.851020 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849645    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-sysconfig\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.851020 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849744    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-lib-modules\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.851020 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849767    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-socket-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.851020 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849809    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-run-netns\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.851020 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849848    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-log-socket\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.851856 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849899    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-host\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.851856 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.849936    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd24c416-9f3c-4955-a536-08f20ccdbbbb-host-slash\") pod \"iptables-alerter-4q28p\" (UID: \"fd24c416-9f3c-4955-a536-08f20ccdbbbb\") " pod="openshift-network-operator/iptables-alerter-4q28p"
Oct 30 19:05:29.851856 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.850196    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fd24c416-9f3c-4955-a536-08f20ccdbbbb-iptables-alerter-script\") pod \"iptables-alerter-4q28p\" (UID: \"fd24c416-9f3c-4955-a536-08f20ccdbbbb\") " pod="openshift-network-operator/iptables-alerter-4q28p"
Oct 30 19:05:29.851856 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.850255    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8d23c4bb-0012-4e09-bc59-c6689f9ea394-konnectivity-ca\") pod \"konnectivity-agent-ln5ql\" (UID: \"8d23c4bb-0012-4e09-bc59-c6689f9ea394\") " pod="kube-system/konnectivity-agent-ln5ql"
Oct 30 19:05:29.851856 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.850300    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-etc-selinux\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.851856 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.850303    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-env-overrides\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.851856 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.850333    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-modprobe-d\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.851856 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.850417    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-dbus\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:29.851856 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.850419    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-sysctl-d\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.851856 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.850448    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-ovnkube-config\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.851856 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.850543    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-var-lib-kubelet\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.851856 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.850918    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-ovnkube-script-lib\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.852453 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.852037    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8d23c4bb-0012-4e09-bc59-c6689f9ea394-agent-certs\") pod \"konnectivity-agent-ln5ql\" (UID: \"8d23c4bb-0012-4e09-bc59-c6689f9ea394\") " pod="kube-system/konnectivity-agent-ln5ql"
Oct 30 19:05:29.852453 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.852211    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-tmp\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.852862 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.852840    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-tuned\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.854335 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.854316    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-ovn-node-metrics-cert\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.859962 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:29.859938    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:05:29.860046 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:29.859969    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:05:29.860046 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:29.859983    2586 projected.go:196] Error preparing data for projected volume kube-api-access-8pq2q for pod openshift-network-diagnostics/network-check-target-z2dbh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:05:29.860181 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:29.860049    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q podName:c1cc12f1-be52-4a9f-a8a2-53fe3c493431 nodeName:}" failed. No retries permitted until 2026-10-30 19:05:30.360032984 +0000 UTC m=+3.084297684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8pq2q" (UniqueName: "kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q") pod "network-check-target-z2dbh" (UID: "c1cc12f1-be52-4a9f-a8a2-53fe3c493431") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:05:29.862623 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.862601    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdsdn\" (UniqueName: \"kubernetes.io/projected/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-kube-api-access-qdsdn\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:29.863780 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.863758    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49bwq\" (UniqueName: \"kubernetes.io/projected/fd24c416-9f3c-4955-a536-08f20ccdbbbb-kube-api-access-49bwq\") pod \"iptables-alerter-4q28p\" (UID: \"fd24c416-9f3c-4955-a536-08f20ccdbbbb\") " pod="openshift-network-operator/iptables-alerter-4q28p"
Oct 30 19:05:29.864535 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.864499    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxvt7\" (UniqueName: \"kubernetes.io/projected/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-kube-api-access-qxvt7\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:29.869810 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.869790    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nlrh\" (UniqueName: \"kubernetes.io/projected/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-kube-api-access-9nlrh\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:29.955762 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.955732    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8jrs8"
Oct 30 19:05:29.957880 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.957847    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:05:29.961889 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.961866    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2z2fd"
Oct 30 19:05:29.964394 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.964377    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lq697"
Oct 30 19:05:29.970905 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:29.970878    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94709da2_704a_421c_aa2f_1844db5bec96.slice/crio-d1edc3f3a2dcdf669197742726ce951eb7f03990d787797f1a4a4c32b9085448 WatchSource:0}: Error finding container d1edc3f3a2dcdf669197742726ce951eb7f03990d787797f1a4a4c32b9085448: Status 404 returned error can't find the container with id d1edc3f3a2dcdf669197742726ce951eb7f03990d787797f1a4a4c32b9085448
Oct 30 19:05:29.974135 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.973439    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4q28p"
Oct 30 19:05:29.974394 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:29.974290    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7c1242a_b12d_49b2_8052_781394f01418.slice/crio-b83b2f591bd7deef664e4e37dbd9804306d3afe4efeb9f587032c95e3e194dfd WatchSource:0}: Error finding container b83b2f591bd7deef664e4e37dbd9804306d3afe4efeb9f587032c95e3e194dfd: Status 404 returned error can't find the container with id b83b2f591bd7deef664e4e37dbd9804306d3afe4efeb9f587032c95e3e194dfd
Oct 30 19:05:29.975618 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:29.975595    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod838f8f5e_ac73_4280_8ed3_173a51c42b9e.slice/crio-a2167fb8c16acfe29480b3b313e7b0c993c3a07c9aa231868e3cfc7c3e909333 WatchSource:0}: Error finding container a2167fb8c16acfe29480b3b313e7b0c993c3a07c9aa231868e3cfc7c3e909333: Status 404 returned error can't find the container with id a2167fb8c16acfe29480b3b313e7b0c993c3a07c9aa231868e3cfc7c3e909333
Oct 30 19:05:29.979011 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:29.978465    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbcf7627_e9b7_49ee_80ff_d2f05c8e9d3a.slice/crio-a3f490ce9eb80385ff59eaf9e69bee4549d03457e691558955ca4e45456ec0d7 WatchSource:0}: Error finding container a3f490ce9eb80385ff59eaf9e69bee4549d03457e691558955ca4e45456ec0d7: Status 404 returned error can't find the container with id a3f490ce9eb80385ff59eaf9e69bee4549d03457e691558955ca4e45456ec0d7
Oct 30 19:05:29.980795 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.980776    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ln5ql"
Oct 30 19:05:29.983809 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:29.983788    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd24c416_9f3c_4955_a536_08f20ccdbbbb.slice/crio-8c20c3f66ae4d81e1612ec8a561e5e595426adad4dead32438fcc369135a765f WatchSource:0}: Error finding container 8c20c3f66ae4d81e1612ec8a561e5e595426adad4dead32438fcc369135a765f: Status 404 returned error can't find the container with id 8c20c3f66ae4d81e1612ec8a561e5e595426adad4dead32438fcc369135a765f
Oct 30 19:05:29.990207 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:29.990185    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d23c4bb_0012_4e09_bc59_c6689f9ea394.slice/crio-bf9224e80094e9d2e45da0443ed7c91b2e99fe2da568a6619c8c63306517c3f7 WatchSource:0}: Error finding container bf9224e80094e9d2e45da0443ed7c91b2e99fe2da568a6619c8c63306517c3f7: Status 404 returned error can't find the container with id bf9224e80094e9d2e45da0443ed7c91b2e99fe2da568a6619c8c63306517c3f7
Oct 30 19:05:29.992430 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:29.992411    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:05:30.002117 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:30.002091    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod260d4781_1e14_4a9a_8840_49c1f9bdbfbe.slice/crio-010e578cb4471120dd639ca96c7e1d98938702738350708dc15596dac065894c WatchSource:0}: Error finding container 010e578cb4471120dd639ca96c7e1d98938702738350708dc15596dac065894c: Status 404 returned error can't find the container with id 010e578cb4471120dd639ca96c7e1d98938702738350708dc15596dac065894c
Oct 30 19:05:30.006989 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:30.006967    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:05:30.012937 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:30.012919    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:30.016752 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:30.016730    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfef7bdd6_e0b1_4a2c_ab40_274408d6d0d1.slice/crio-dcaacf3a4d0a3883ab40fc083c56e4d06ae2f567d9495711f47fe108cb811c20 WatchSource:0}: Error finding container dcaacf3a4d0a3883ab40fc083c56e4d06ae2f567d9495711f47fe108cb811c20: Status 404 returned error can't find the container with id dcaacf3a4d0a3883ab40fc083c56e4d06ae2f567d9495711f47fe108cb811c20
Oct 30 19:05:30.023889 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:30.023863    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30fc2aa0_31d5_4577_8064_25dbb6cea5d7.slice/crio-5f0fc158d89f6d16c311fafc38d332790bdbfda0144023ef8074b2a4aeaa76a5 WatchSource:0}: Error finding container 5f0fc158d89f6d16c311fafc38d332790bdbfda0144023ef8074b2a4aeaa76a5: Status 404 returned error can't find the container with id 5f0fc158d89f6d16c311fafc38d332790bdbfda0144023ef8074b2a4aeaa76a5
Oct 30 19:05:30.253925 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:30.253851    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs\") pod \"network-metrics-daemon-xmkt2\" (UID: \"ed4e7b61-d816-49fc-b4d9-36236a716227\") " pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:30.254060 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:30.253989    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:05:30.254060 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:30.254047    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs podName:ed4e7b61-d816-49fc-b4d9-36236a716227 nodeName:}" failed. No retries permitted until 2026-10-30 19:05:31.254030424 +0000 UTC m=+3.978295124 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs") pod "network-metrics-daemon-xmkt2" (UID: "ed4e7b61-d816-49fc-b4d9-36236a716227") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:05:30.354521 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:30.354428    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:30.354610 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:30.354591    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:05:30.354672 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:30.354661    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret podName:ba82c577-b89d-44f7-87e4-2d00d1e2a5de nodeName:}" failed. No retries permitted until 2026-10-30 19:05:31.354641146 +0000 UTC m=+4.078905857 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret") pod "global-pull-secret-syncer-7q5vl" (UID: "ba82c577-b89d-44f7-87e4-2d00d1e2a5de") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:05:30.456435 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:30.456403    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pq2q\" (UniqueName: \"kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q\") pod \"network-check-target-z2dbh\" (UID: \"c1cc12f1-be52-4a9f-a8a2-53fe3c493431\") " pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:30.456629 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:30.456613    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:05:30.456696 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:30.456640    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:05:30.456696 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:30.456653    2586 projected.go:196] Error preparing data for projected volume kube-api-access-8pq2q for pod openshift-network-diagnostics/network-check-target-z2dbh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:05:30.456880 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:30.456758    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q podName:c1cc12f1-be52-4a9f-a8a2-53fe3c493431 nodeName:}" failed. No retries permitted until 2026-10-30 19:05:31.456700485 +0000 UTC m=+4.180965187 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8pq2q" (UniqueName: "kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q") pod "network-check-target-z2dbh" (UID: "c1cc12f1-be52-4a9f-a8a2-53fe3c493431") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:05:30.680464 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:30.680388    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:00:29 +0000 UTC" deadline="2028-06-27 23:54:04.461083446 +0000 UTC"
Oct 30 19:05:30.680464 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:30.680461    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14548h48m33.780625563s"
Oct 30 19:05:30.711651 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:30.711330    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal" event={"ID":"7e14d55fcbb7a4643b4d48f7935b8114","Type":"ContainerStarted","Data":"817e40d960fc2832d270c769b54faddea0c8e02198c1b6a8887c27cc2fce352a"}
Oct 30 19:05:30.712990 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:30.712900    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw" event={"ID":"30fc2aa0-31d5-4577-8064-25dbb6cea5d7","Type":"ContainerStarted","Data":"5f0fc158d89f6d16c311fafc38d332790bdbfda0144023ef8074b2a4aeaa76a5"}
Oct 30 19:05:30.714452 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:30.714401    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4q28p" event={"ID":"fd24c416-9f3c-4955-a536-08f20ccdbbbb","Type":"ContainerStarted","Data":"8c20c3f66ae4d81e1612ec8a561e5e595426adad4dead32438fcc369135a765f"}
Oct 30 19:05:30.717513 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:30.716937    2586 generic.go:356] "Generic (PLEG): container finished" podID="7fe3dcaf32fbbd51ccedffe35a8755ab" containerID="a346e3a69fec2fa80723317b10070413f8ab9ee567a5cf9d2f8c9b22f59a303d" exitCode=0
Oct 30 19:05:30.717513 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:30.717003    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal" event={"ID":"7fe3dcaf32fbbd51ccedffe35a8755ab","Type":"ContainerDied","Data":"a346e3a69fec2fa80723317b10070413f8ab9ee567a5cf9d2f8c9b22f59a303d"}
Oct 30 19:05:30.721188 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:30.721106    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b7bcf" event={"ID":"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1","Type":"ContainerStarted","Data":"dcaacf3a4d0a3883ab40fc083c56e4d06ae2f567d9495711f47fe108cb811c20"}
Oct 30 19:05:30.734047 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:30.731982    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal" podStartSLOduration=1.7319671319999999 podStartE2EDuration="1.731967132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:05:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:30.731480501 +0000 UTC m=+3.455745217" watchObservedRunningTime="2026-10-30 19:05:30.731967132 +0000 UTC m=+3.456231851"
Oct 30 19:05:30.736608 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:30.736499    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn" event={"ID":"260d4781-1e14-4a9a-8840-49c1f9bdbfbe","Type":"ContainerStarted","Data":"010e578cb4471120dd639ca96c7e1d98938702738350708dc15596dac065894c"}
Oct 30 19:05:30.738613 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:30.738539    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ln5ql" event={"ID":"8d23c4bb-0012-4e09-bc59-c6689f9ea394","Type":"ContainerStarted","Data":"bf9224e80094e9d2e45da0443ed7c91b2e99fe2da568a6619c8c63306517c3f7"}
Oct 30 19:05:30.741926 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:30.741899    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lq697" event={"ID":"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a","Type":"ContainerStarted","Data":"a3f490ce9eb80385ff59eaf9e69bee4549d03457e691558955ca4e45456ec0d7"}
Oct 30 19:05:30.744357 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:30.744303    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2z2fd" event={"ID":"838f8f5e-ac73-4280-8ed3-173a51c42b9e","Type":"ContainerStarted","Data":"a2167fb8c16acfe29480b3b313e7b0c993c3a07c9aa231868e3cfc7c3e909333"}
Oct 30 19:05:30.761338 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:30.761251    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d2ntn" event={"ID":"b7c1242a-b12d-49b2-8052-781394f01418","Type":"ContainerStarted","Data":"b83b2f591bd7deef664e4e37dbd9804306d3afe4efeb9f587032c95e3e194dfd"}
Oct 30 19:05:30.774300 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:30.774067    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8jrs8" event={"ID":"94709da2-704a-421c-aa2f-1844db5bec96","Type":"ContainerStarted","Data":"d1edc3f3a2dcdf669197742726ce951eb7f03990d787797f1a4a4c32b9085448"}
Oct 30 19:05:31.264200 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:31.264149    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs\") pod \"network-metrics-daemon-xmkt2\" (UID: \"ed4e7b61-d816-49fc-b4d9-36236a716227\") " pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:31.264393 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:31.264308    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:05:31.264393 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:31.264368    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs podName:ed4e7b61-d816-49fc-b4d9-36236a716227 nodeName:}" failed. No retries permitted until 2026-10-30 19:05:33.264351567 +0000 UTC m=+5.988616270 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs") pod "network-metrics-daemon-xmkt2" (UID: "ed4e7b61-d816-49fc-b4d9-36236a716227") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:05:31.364989 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:31.364950    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:31.365157 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:31.365141    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:05:31.365253 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:31.365221    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret podName:ba82c577-b89d-44f7-87e4-2d00d1e2a5de nodeName:}" failed. No retries permitted until 2026-10-30 19:05:33.365203 +0000 UTC m=+6.089467708 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret") pod "global-pull-secret-syncer-7q5vl" (UID: "ba82c577-b89d-44f7-87e4-2d00d1e2a5de") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:05:31.465361 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:31.465321    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pq2q\" (UniqueName: \"kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q\") pod \"network-check-target-z2dbh\" (UID: \"c1cc12f1-be52-4a9f-a8a2-53fe3c493431\") " pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:31.465514 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:31.465497    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:05:31.465564 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:31.465523    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:05:31.465564 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:31.465537    2586 projected.go:196] Error preparing data for projected volume kube-api-access-8pq2q for pod openshift-network-diagnostics/network-check-target-z2dbh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:05:31.465625 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:31.465590    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q podName:c1cc12f1-be52-4a9f-a8a2-53fe3c493431 nodeName:}" failed. No retries permitted until 2026-10-30 19:05:33.465574421 +0000 UTC m=+6.189839131 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8pq2q" (UniqueName: "kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q") pod "network-check-target-z2dbh" (UID: "c1cc12f1-be52-4a9f-a8a2-53fe3c493431") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:05:31.681004 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:31.680934    2586 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:00:29 +0000 UTC" deadline="2028-07-24 12:45:03.82815138 +0000 UTC"
Oct 30 19:05:31.681004 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:31.680973    2586 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15185h39m32.147183171s"
Oct 30 19:05:31.703548 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:31.702734    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:31.703548 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:31.702857    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q5vl" podUID="ba82c577-b89d-44f7-87e4-2d00d1e2a5de"
Oct 30 19:05:31.703548 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:31.703257    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:31.703548 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:31.703361    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmkt2" podUID="ed4e7b61-d816-49fc-b4d9-36236a716227"
Oct 30 19:05:31.703548 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:31.703430    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:31.703548 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:31.703496    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2dbh" podUID="c1cc12f1-be52-4a9f-a8a2-53fe3c493431"
Oct 30 19:05:31.790510 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:31.790442    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal" event={"ID":"7fe3dcaf32fbbd51ccedffe35a8755ab","Type":"ContainerStarted","Data":"5150f5b5eac32390f7988cd29576fec52fbefb1c90f35f39dd0e2512990d398f"}
Oct 30 19:05:33.284730 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:33.284644    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs\") pod \"network-metrics-daemon-xmkt2\" (UID: \"ed4e7b61-d816-49fc-b4d9-36236a716227\") " pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:33.288773 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:33.288743    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:05:33.288890 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:33.288845    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs podName:ed4e7b61-d816-49fc-b4d9-36236a716227 nodeName:}" failed. No retries permitted until 2026-10-30 19:05:37.288816541 +0000 UTC m=+10.013081236 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs") pod "network-metrics-daemon-xmkt2" (UID: "ed4e7b61-d816-49fc-b4d9-36236a716227") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:05:33.385679 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:33.385628    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:33.385835 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:33.385801    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:05:33.385916 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:33.385901    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret podName:ba82c577-b89d-44f7-87e4-2d00d1e2a5de nodeName:}" failed. No retries permitted until 2026-10-30 19:05:37.385859802 +0000 UTC m=+10.110124502 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret") pod "global-pull-secret-syncer-7q5vl" (UID: "ba82c577-b89d-44f7-87e4-2d00d1e2a5de") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:05:33.486722 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:33.486680    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pq2q\" (UniqueName: \"kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q\") pod \"network-check-target-z2dbh\" (UID: \"c1cc12f1-be52-4a9f-a8a2-53fe3c493431\") " pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:33.486892 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:33.486847    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:05:33.486892 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:33.486870    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:05:33.486892 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:33.486883    2586 projected.go:196] Error preparing data for projected volume kube-api-access-8pq2q for pod openshift-network-diagnostics/network-check-target-z2dbh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:05:33.487053 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:33.486937    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q podName:c1cc12f1-be52-4a9f-a8a2-53fe3c493431 nodeName:}" failed. No retries permitted until 2026-10-30 19:05:37.486919891 +0000 UTC m=+10.211184592 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8pq2q" (UniqueName: "kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q") pod "network-check-target-z2dbh" (UID: "c1cc12f1-be52-4a9f-a8a2-53fe3c493431") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:05:33.703383 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:33.703340    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:33.703544 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:33.703363    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:33.703544 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:33.703475    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2dbh" podUID="c1cc12f1-be52-4a9f-a8a2-53fe3c493431"
Oct 30 19:05:33.703544 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:33.703507    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:33.703710 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:33.703611    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmkt2" podUID="ed4e7b61-d816-49fc-b4d9-36236a716227"
Oct 30 19:05:33.703710 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:33.703698    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q5vl" podUID="ba82c577-b89d-44f7-87e4-2d00d1e2a5de"
Oct 30 19:05:33.815673 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:33.815638    2586 generic.go:356] "Generic (PLEG): container finished" podID="b7c1242a-b12d-49b2-8052-781394f01418" containerID="4325ed0d6ea13153968c7f0c587f987f6a6ee5f21ff8886fd8ae09f566134612" exitCode=0
Oct 30 19:05:33.815842 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:33.815705    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d2ntn" event={"ID":"b7c1242a-b12d-49b2-8052-781394f01418","Type":"ContainerDied","Data":"4325ed0d6ea13153968c7f0c587f987f6a6ee5f21ff8886fd8ae09f566134612"}
Oct 30 19:05:33.844566 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:33.843561    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal" podStartSLOduration=4.843544784 podStartE2EDuration="4.843544784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:05:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:31.809555193 +0000 UTC m=+4.533819911" watchObservedRunningTime="2026-10-30 19:05:33.843544784 +0000 UTC m=+6.567809504"
Oct 30 19:05:34.829198 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:34.828742    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ln5ql" event={"ID":"8d23c4bb-0012-4e09-bc59-c6689f9ea394","Type":"ContainerStarted","Data":"8ee948f44797f312a2796125e041319624d2f6e760dfd885cf91b0015770dab7"}
Oct 30 19:05:34.849186 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:34.849126    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ln5ql" podStartSLOduration=4.123985735 podStartE2EDuration="7.849114437s" totalImagesPullingTime="3.725128702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:34.849087457 +0000 UTC m=+7.573352174" watchObservedRunningTime="2026-10-30 19:05:34.849114437 +0000 UTC m=+7.573379153"
Oct 30 19:05:35.701298 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:35.701268    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:35.701478 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:35.701280    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:35.701478 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:35.701412    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmkt2" podUID="ed4e7b61-d816-49fc-b4d9-36236a716227"
Oct 30 19:05:35.701478 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:35.701456    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:35.701681 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:35.701559    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2dbh" podUID="c1cc12f1-be52-4a9f-a8a2-53fe3c493431"
Oct 30 19:05:35.701681 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:35.701653    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q5vl" podUID="ba82c577-b89d-44f7-87e4-2d00d1e2a5de"
Oct 30 19:05:35.844410 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:35.844338    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn" event={"ID":"260d4781-1e14-4a9a-8840-49c1f9bdbfbe","Type":"ContainerStarted","Data":"3684356cb1a55744a11670ba270cde21c77fbd8443a131d712d09238d416c238"}
Oct 30 19:05:35.847730 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:35.847700    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2z2fd" event={"ID":"838f8f5e-ac73-4280-8ed3-173a51c42b9e","Type":"ContainerStarted","Data":"d6551730b2628f9c5ecc211e7e5b3f019c06dac7696ab819b1c4294d285b8e65"}
Oct 30 19:05:35.879399 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:35.879347    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2z2fd" podStartSLOduration=4.169532388 podStartE2EDuration="8.879330142s" totalImagesPullingTime="4.709797754s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:05:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:35.878783132 +0000 UTC m=+8.603047849" watchObservedRunningTime="2026-10-30 19:05:35.879330142 +0000 UTC m=+8.603594863"
Oct 30 19:05:36.822123 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:36.822087    2586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ln5ql"
Oct 30 19:05:36.822789 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:36.822771    2586 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ln5ql"
Oct 30 19:05:36.878519 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:36.878495    2586 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 19:05:37.318480 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:37.318449    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs\") pod \"network-metrics-daemon-xmkt2\" (UID: \"ed4e7b61-d816-49fc-b4d9-36236a716227\") " pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:37.318657 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:37.318603    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:05:37.318728 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:37.318669    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs podName:ed4e7b61-d816-49fc-b4d9-36236a716227 nodeName:}" failed. No retries permitted until 2026-10-30 19:05:45.318649994 +0000 UTC m=+18.042914702 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs") pod "network-metrics-daemon-xmkt2" (UID: "ed4e7b61-d816-49fc-b4d9-36236a716227") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:05:37.419631 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:37.419106    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:37.419631 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:37.419287    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:05:37.419631 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:37.419349    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret podName:ba82c577-b89d-44f7-87e4-2d00d1e2a5de nodeName:}" failed. No retries permitted until 2026-10-30 19:05:45.419332094 +0000 UTC m=+18.143596790 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret") pod "global-pull-secret-syncer-7q5vl" (UID: "ba82c577-b89d-44f7-87e4-2d00d1e2a5de") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:05:37.520331 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:37.520279    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pq2q\" (UniqueName: \"kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q\") pod \"network-check-target-z2dbh\" (UID: \"c1cc12f1-be52-4a9f-a8a2-53fe3c493431\") " pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:37.520501 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:37.520450    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:05:37.520501 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:37.520471    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:05:37.520501 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:37.520485    2586 projected.go:196] Error preparing data for projected volume kube-api-access-8pq2q for pod openshift-network-diagnostics/network-check-target-z2dbh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:05:37.520663 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:37.520535    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q podName:c1cc12f1-be52-4a9f-a8a2-53fe3c493431 nodeName:}" failed. No retries permitted until 2026-10-30 19:05:45.520522448 +0000 UTC m=+18.244787170 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8pq2q" (UniqueName: "kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q") pod "network-check-target-z2dbh" (UID: "c1cc12f1-be52-4a9f-a8a2-53fe3c493431") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:05:37.695575 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:37.695235    2586 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T19:05:36.878515743Z","UUID":"4500e874-2e6f-4cba-9634-a96d749caed2","Handler":null,"Name":"","Endpoint":""}
Oct 30 19:05:37.697592 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:37.697140    2586 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 19:05:37.697592 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:37.697184    2586 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 19:05:37.704635 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:37.703808    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:37.704635 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:37.703957    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2dbh" podUID="c1cc12f1-be52-4a9f-a8a2-53fe3c493431"
Oct 30 19:05:37.704635 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:37.704350    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:37.704635 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:37.704441    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q5vl" podUID="ba82c577-b89d-44f7-87e4-2d00d1e2a5de"
Oct 30 19:05:37.705120 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:37.704966    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:37.705120 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:37.705068    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmkt2" podUID="ed4e7b61-d816-49fc-b4d9-36236a716227"
Oct 30 19:05:37.855394 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:37.855356    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8jrs8" event={"ID":"94709da2-704a-421c-aa2f-1844db5bec96","Type":"ContainerStarted","Data":"17b70c8321b442cbb5cb0a4a9b351d83ec6b01432b345b6f1578339de7a9f9d0"}
Oct 30 19:05:37.858500 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:37.858426    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn" event={"ID":"260d4781-1e14-4a9a-8840-49c1f9bdbfbe","Type":"ContainerStarted","Data":"2feab4534e9c7962469a84128977f8eb60983377b137037a9c1675c676d15705"}
Oct 30 19:05:37.862477 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:37.862455    2586 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:05:37.862876 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:37.862856    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lq697" event={"ID":"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a","Type":"ContainerStarted","Data":"f02625a0259bd9484517db441b305fad6d03146cbc42f79f9e3998f039b65711"}
Oct 30 19:05:37.909869 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:37.908963    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-8jrs8" podStartSLOduration=3.30877189 podStartE2EDuration="10.908947786s" totalImagesPullingTime="7.600175896s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:05:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:37.875916223 +0000 UTC m=+10.600180949" watchObservedRunningTime="2026-10-30 19:05:37.908947786 +0000 UTC m=+10.633212504"
Oct 30 19:05:38.867390 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:38.867125    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn" event={"ID":"260d4781-1e14-4a9a-8840-49c1f9bdbfbe","Type":"ContainerStarted","Data":"bba3733063132fc597c6cda795d5cbe13eb6708e406f093fc41985d32bf88337"}
Oct 30 19:05:38.869343 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:38.869313    2586 generic.go:356] "Generic (PLEG): container finished" podID="b7c1242a-b12d-49b2-8052-781394f01418" containerID="5174e163852289ef2d62bd9c6643e73db8259644771c17b4ebc807fad7359667" exitCode=0
Oct 30 19:05:38.869469 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:38.869365    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d2ntn" event={"ID":"b7c1242a-b12d-49b2-8052-781394f01418","Type":"ContainerDied","Data":"5174e163852289ef2d62bd9c6643e73db8259644771c17b4ebc807fad7359667"}
Oct 30 19:05:38.889695 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:38.889656    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn" podStartSLOduration=4.098509759 podStartE2EDuration="11.889644909s" totalImagesPullingTime="7.79113515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:05:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:38.889447012 +0000 UTC m=+11.613711728" watchObservedRunningTime="2026-10-30 19:05:38.889644909 +0000 UTC m=+11.613909626"
Oct 30 19:05:38.889837 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:38.889810    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-lq697" podStartSLOduration=4.500392675 podStartE2EDuration="11.889805496s" totalImagesPullingTime="7.389412821s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:05:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:37.910517259 +0000 UTC m=+10.634781976" watchObservedRunningTime="2026-10-30 19:05:38.889805496 +0000 UTC m=+11.614070212"
Oct 30 19:05:39.700553 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:39.700521    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:39.701011 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:39.700585    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:39.701011 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:39.700590    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:39.701011 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:39.700697    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmkt2" podUID="ed4e7b61-d816-49fc-b4d9-36236a716227"
Oct 30 19:05:39.701011 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:39.700817    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q5vl" podUID="ba82c577-b89d-44f7-87e4-2d00d1e2a5de"
Oct 30 19:05:39.701011 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:39.700900    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2dbh" podUID="c1cc12f1-be52-4a9f-a8a2-53fe3c493431"
Oct 30 19:05:39.873499 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:39.872928    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4q28p" event={"ID":"fd24c416-9f3c-4955-a536-08f20ccdbbbb","Type":"ContainerStarted","Data":"c0fa74bf627f607c2c4009396304b9bde38525ee2fd5f1d2c2f7aca06a9233fd"}
Oct 30 19:05:39.890900 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:39.890850    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4q28p" podStartSLOduration=4.900051802 podStartE2EDuration="12.890833768s" totalImagesPullingTime="7.990781966s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:05:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:39.890539431 +0000 UTC m=+12.614804148" watchObservedRunningTime="2026-10-30 19:05:39.890833768 +0000 UTC m=+12.615098502"
Oct 30 19:05:40.876490 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:40.876451    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-b7bcf" event={"ID":"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1","Type":"ContainerStarted","Data":"368eb310c60d3113fc5c041fd5aa607c0696a41d7e584fe9b05802c0d56b9d4f"}
Oct 30 19:05:40.879491 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:40.879452    2586 generic.go:356] "Generic (PLEG): container finished" podID="b7c1242a-b12d-49b2-8052-781394f01418" containerID="35288109388ddbffbde76f0cac03fa3a0d4c11a1eaca766e19ee65b800f68025" exitCode=0
Oct 30 19:05:40.879612 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:40.879541    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d2ntn" event={"ID":"b7c1242a-b12d-49b2-8052-781394f01418","Type":"ContainerDied","Data":"35288109388ddbffbde76f0cac03fa3a0d4c11a1eaca766e19ee65b800f68025"}
Oct 30 19:05:40.897196 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:40.897139    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-b7bcf" podStartSLOduration=3.822741676 podStartE2EDuration="13.897124657s" totalImagesPullingTime="10.074382981s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:05:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:40.896683525 +0000 UTC m=+13.620948243" watchObservedRunningTime="2026-10-30 19:05:40.897124657 +0000 UTC m=+13.621389376"
Oct 30 19:05:41.700912 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:41.700885    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:41.701028 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:41.701005    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q5vl" podUID="ba82c577-b89d-44f7-87e4-2d00d1e2a5de"
Oct 30 19:05:41.701409 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:41.701387    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:41.701530 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:41.701498    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmkt2" podUID="ed4e7b61-d816-49fc-b4d9-36236a716227"
Oct 30 19:05:41.701649 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:41.701633    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:41.701736 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:41.701715    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2dbh" podUID="c1cc12f1-be52-4a9f-a8a2-53fe3c493431"
Oct 30 19:05:41.884586 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:41.884557    2586 generic.go:356] "Generic (PLEG): container finished" podID="b7c1242a-b12d-49b2-8052-781394f01418" containerID="a273df99c005f00ff6a2ec5561ba922b7c712ab10b7cef763ca9b006d6556c38" exitCode=0
Oct 30 19:05:41.885183 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:41.884622    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d2ntn" event={"ID":"b7c1242a-b12d-49b2-8052-781394f01418","Type":"ContainerDied","Data":"a273df99c005f00ff6a2ec5561ba922b7c712ab10b7cef763ca9b006d6556c38"}
Oct 30 19:05:43.700961 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:43.700761    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:43.701429 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:43.700776    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:43.701429 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:43.701060    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2dbh" podUID="c1cc12f1-be52-4a9f-a8a2-53fe3c493431"
Oct 30 19:05:43.701429 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:43.700832    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:43.701429 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:43.701163    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmkt2" podUID="ed4e7b61-d816-49fc-b4d9-36236a716227"
Oct 30 19:05:43.701429 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:43.701299    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q5vl" podUID="ba82c577-b89d-44f7-87e4-2d00d1e2a5de"
Oct 30 19:05:44.894062 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:44.893850    2586 generic.go:356] "Generic (PLEG): container finished" podID="b7c1242a-b12d-49b2-8052-781394f01418" containerID="56c27dd8b5c096104c275e78ac2b79f65f2806f0dde5bce846d11c4f2a0ce4fe" exitCode=0
Oct 30 19:05:44.894439 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:44.893906    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d2ntn" event={"ID":"b7c1242a-b12d-49b2-8052-781394f01418","Type":"ContainerDied","Data":"56c27dd8b5c096104c275e78ac2b79f65f2806f0dde5bce846d11c4f2a0ce4fe"}
Oct 30 19:05:44.896868 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:44.896843    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw" event={"ID":"30fc2aa0-31d5-4577-8064-25dbb6cea5d7","Type":"ContainerStarted","Data":"e0890e7735aa77896f68129dddd9b0b6ecec9b069b4691ed303bf23c0e3b7a06"}
Oct 30 19:05:44.896943 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:44.896923    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw" event={"ID":"30fc2aa0-31d5-4577-8064-25dbb6cea5d7","Type":"ContainerStarted","Data":"6a8203114d71c9a95d69b03afb5bf3c66b4424b422d101186bf6280376b3cc47"}
Oct 30 19:05:44.896988 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:44.896975    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw" event={"ID":"30fc2aa0-31d5-4577-8064-25dbb6cea5d7","Type":"ContainerStarted","Data":"fb30e95531fdb1311d5635c29b5f82e87684810541530a82d8ad0ae3e5078196"}
Oct 30 19:05:44.897042 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:44.897029    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw" event={"ID":"30fc2aa0-31d5-4577-8064-25dbb6cea5d7","Type":"ContainerStarted","Data":"6cb83bc4c3f99209f3a9cfd435f6f257d4656dea9a104a8a414f5c4c646c22fd"}
Oct 30 19:05:44.897100 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:44.897088    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw" event={"ID":"30fc2aa0-31d5-4577-8064-25dbb6cea5d7","Type":"ContainerStarted","Data":"97023662ba03d5e8c72e251399ebfaad0a3adc6379fc9312bd9cc65c14e34441"}
Oct 30 19:05:45.384292 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:45.384261    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs\") pod \"network-metrics-daemon-xmkt2\" (UID: \"ed4e7b61-d816-49fc-b4d9-36236a716227\") " pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:45.384454 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:45.384384    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:05:45.384454 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:45.384439    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs podName:ed4e7b61-d816-49fc-b4d9-36236a716227 nodeName:}" failed. No retries permitted until 2026-10-30 19:06:01.384420709 +0000 UTC m=+34.108685404 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs") pod "network-metrics-daemon-xmkt2" (UID: "ed4e7b61-d816-49fc-b4d9-36236a716227") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 19:05:45.485264 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:45.485234    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:45.485389 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:45.485334    2586 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 19:05:45.485389 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:45.485374    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret podName:ba82c577-b89d-44f7-87e4-2d00d1e2a5de nodeName:}" failed. No retries permitted until 2026-10-30 19:06:01.485363885 +0000 UTC m=+34.209628580 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret") pod "global-pull-secret-syncer-7q5vl" (UID: "ba82c577-b89d-44f7-87e4-2d00d1e2a5de") : object "kube-system"/"original-pull-secret" not registered
Oct 30 19:05:45.586187 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:45.586154    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pq2q\" (UniqueName: \"kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q\") pod \"network-check-target-z2dbh\" (UID: \"c1cc12f1-be52-4a9f-a8a2-53fe3c493431\") " pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:45.586339 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:45.586319    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 19:05:45.586403 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:45.586350    2586 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 19:05:45.586403 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:45.586364    2586 projected.go:196] Error preparing data for projected volume kube-api-access-8pq2q for pod openshift-network-diagnostics/network-check-target-z2dbh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:05:45.586490 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:45.586419    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q podName:c1cc12f1-be52-4a9f-a8a2-53fe3c493431 nodeName:}" failed. No retries permitted until 2026-10-30 19:06:01.586402237 +0000 UTC m=+34.310666931 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8pq2q" (UniqueName: "kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q") pod "network-check-target-z2dbh" (UID: "c1cc12f1-be52-4a9f-a8a2-53fe3c493431") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 19:05:45.700889 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:45.700831    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:45.701007 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:45.700831    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:45.701007 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:45.700936    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:45.701007 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:45.700932    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2dbh" podUID="c1cc12f1-be52-4a9f-a8a2-53fe3c493431"
Oct 30 19:05:45.701147 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:45.701033    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmkt2" podUID="ed4e7b61-d816-49fc-b4d9-36236a716227"
Oct 30 19:05:45.701147 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:45.701121    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q5vl" podUID="ba82c577-b89d-44f7-87e4-2d00d1e2a5de"
Oct 30 19:05:45.901267 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:45.901233    2586 generic.go:356] "Generic (PLEG): container finished" podID="b7c1242a-b12d-49b2-8052-781394f01418" containerID="15291a49719c9ee66421e9994d9db770b669de936127bb7d60a87ea1935c5583" exitCode=0
Oct 30 19:05:45.901761 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:45.901317    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d2ntn" event={"ID":"b7c1242a-b12d-49b2-8052-781394f01418","Type":"ContainerDied","Data":"15291a49719c9ee66421e9994d9db770b669de936127bb7d60a87ea1935c5583"}
Oct 30 19:05:45.904343 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:45.904324    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw" event={"ID":"30fc2aa0-31d5-4577-8064-25dbb6cea5d7","Type":"ContainerStarted","Data":"219e34f9f3a966dc686ca8f7a7047f88efa42927f17513364f05627c7ce7808c"}
Oct 30 19:05:46.909118 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:46.909087    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d2ntn" event={"ID":"b7c1242a-b12d-49b2-8052-781394f01418","Type":"ContainerStarted","Data":"3d1708c96a8f112ee9cea0f8439505888f90dceeff7da6bfb514eb917dc2ee2b"}
Oct 30 19:05:46.941317 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:46.941276    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-d2ntn" podStartSLOduration=8.903748965 podStartE2EDuration="19.941263262s" totalImagesPullingTime="11.037514297s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:05:27 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:46.941091771 +0000 UTC m=+19.665356487" watchObservedRunningTime="2026-10-30 19:05:46.941263262 +0000 UTC m=+19.665527980"
Oct 30 19:05:47.701336 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:47.701310    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:47.701494 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:47.701403    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q5vl" podUID="ba82c577-b89d-44f7-87e4-2d00d1e2a5de"
Oct 30 19:05:47.701494 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:47.701420    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:47.701494 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:47.701450    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:47.701677 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:47.701531    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmkt2" podUID="ed4e7b61-d816-49fc-b4d9-36236a716227"
Oct 30 19:05:47.701677 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:47.701616    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2dbh" podUID="c1cc12f1-be52-4a9f-a8a2-53fe3c493431"
Oct 30 19:05:47.913542 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:47.913505    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw" event={"ID":"30fc2aa0-31d5-4577-8064-25dbb6cea5d7","Type":"ContainerStarted","Data":"57fe6d5dd526859091e417a372ba874607cbac3f641081f727c6377006d601b4"}
Oct 30 19:05:49.700500 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:49.700469    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:49.701047 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:49.700506    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:49.701047 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:49.700590    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmkt2" podUID="ed4e7b61-d816-49fc-b4d9-36236a716227"
Oct 30 19:05:49.701047 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:49.700642    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:49.701047 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:49.700716    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q5vl" podUID="ba82c577-b89d-44f7-87e4-2d00d1e2a5de"
Oct 30 19:05:49.701047 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:49.700787    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2dbh" podUID="c1cc12f1-be52-4a9f-a8a2-53fe3c493431"
Oct 30 19:05:49.919665 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:49.919641    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw" event={"ID":"30fc2aa0-31d5-4577-8064-25dbb6cea5d7","Type":"ContainerStarted","Data":"b36a4cf33b125af5b63368f4a62580d6756e3b5f01dc8aa1e3f21f2a21a70d57"}
Oct 30 19:05:49.919941 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:49.919917    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:49.935527 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:49.935506    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:49.955851 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:49.955787    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw" podStartSLOduration=8.345547057 podStartE2EDuration="22.955775551s" totalImagesPullingTime="14.610228494s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:05:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:49.953661725 +0000 UTC m=+22.677926443" watchObservedRunningTime="2026-10-30 19:05:49.955775551 +0000 UTC m=+22.680040295"
Oct 30 19:05:50.786764 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:50.786682    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7q5vl"]
Oct 30 19:05:50.787212 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:50.786859    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:50.787212 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:50.786987    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q5vl" podUID="ba82c577-b89d-44f7-87e4-2d00d1e2a5de"
Oct 30 19:05:50.789862 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:50.789830    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z2dbh"]
Oct 30 19:05:50.789993 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:50.789940    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:50.790056 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:50.790040    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2dbh" podUID="c1cc12f1-be52-4a9f-a8a2-53fe3c493431"
Oct 30 19:05:50.790434 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:50.790413    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xmkt2"]
Oct 30 19:05:50.790532 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:50.790505    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:50.790601 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:50.790583    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmkt2" podUID="ed4e7b61-d816-49fc-b4d9-36236a716227"
Oct 30 19:05:50.921840 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:50.921813    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:50.921959 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:50.921868    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:50.936982 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:50.936961    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:05:52.700927 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:52.700896    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:52.701441 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:52.700931    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:52.701441 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:52.700986    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmkt2" podUID="ed4e7b61-d816-49fc-b4d9-36236a716227"
Oct 30 19:05:52.701441 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:52.701013    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:52.701441 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:52.701088    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2dbh" podUID="c1cc12f1-be52-4a9f-a8a2-53fe3c493431"
Oct 30 19:05:52.701441 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:52.701163    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q5vl" podUID="ba82c577-b89d-44f7-87e4-2d00d1e2a5de"
Oct 30 19:05:52.853688 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:52.853665    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ln5ql"
Oct 30 19:05:52.853813 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:52.853760    2586 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:05:52.854110 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:52.854098    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ln5ql"
Oct 30 19:05:54.700963 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:54.700923    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:54.700963 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:54.700951    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:54.701509 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:54.701039    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:54.701509 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:54.701042    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmkt2" podUID="ed4e7b61-d816-49fc-b4d9-36236a716227"
Oct 30 19:05:54.701509 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:54.701112    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q5vl" podUID="ba82c577-b89d-44f7-87e4-2d00d1e2a5de"
Oct 30 19:05:54.701509 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:54.701184    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2dbh" podUID="c1cc12f1-be52-4a9f-a8a2-53fe3c493431"
Oct 30 19:05:56.701320 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:56.701294    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:56.701678 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:56.701375    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:56.701678 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:56.701380    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2dbh" podUID="c1cc12f1-be52-4a9f-a8a2-53fe3c493431"
Oct 30 19:05:56.701678 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:56.701473    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xmkt2" podUID="ed4e7b61-d816-49fc-b4d9-36236a716227"
Oct 30 19:05:56.701678 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:56.701382    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:56.701678 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:56.701544    2586 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7q5vl" podUID="ba82c577-b89d-44f7-87e4-2d00d1e2a5de"
Oct 30 19:05:58.096968 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.096804    2586 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeReady"
Oct 30 19:05:58.097451 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.097066    2586 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 19:05:58.154643 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.154609    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xtj4p"]
Oct 30 19:05:58.211964 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.211936    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-v86zw"]
Oct 30 19:05:58.212124 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.212074    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xtj4p"
Oct 30 19:05:58.216210 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.216191    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-xxr6f\""
Oct 30 19:05:58.216373 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.216357    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 19:05:58.216419 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.216360    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 19:05:58.216457 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.216430    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 19:05:58.232244 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.232211    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xtj4p"]
Oct 30 19:05:58.232244 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.232251    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v86zw"]
Oct 30 19:05:58.232368 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.232305    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v86zw"
Oct 30 19:05:58.273973 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.273948    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7411c16b-533b-4152-bbe6-52262afa37da-cert\") pod \"ingress-canary-xtj4p\" (UID: \"7411c16b-533b-4152-bbe6-52262afa37da\") " pod="openshift-ingress-canary/ingress-canary-xtj4p"
Oct 30 19:05:58.274085 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.274071    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc2g2\" (UniqueName: \"kubernetes.io/projected/7411c16b-533b-4152-bbe6-52262afa37da-kube-api-access-wc2g2\") pod \"ingress-canary-xtj4p\" (UID: \"7411c16b-533b-4152-bbe6-52262afa37da\") " pod="openshift-ingress-canary/ingress-canary-xtj4p"
Oct 30 19:05:58.317502 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.317477    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 19:05:58.317502 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.317477    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 19:05:58.317665 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.317653    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6h85f\""
Oct 30 19:05:58.374498 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.374450    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7411c16b-533b-4152-bbe6-52262afa37da-cert\") pod \"ingress-canary-xtj4p\" (UID: \"7411c16b-533b-4152-bbe6-52262afa37da\") " pod="openshift-ingress-canary/ingress-canary-xtj4p"
Oct 30 19:05:58.374498 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.374479    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce07e9b3-5e98-4d16-8158-afba6fe42281-config-volume\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:05:58.374498 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.374494    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce07e9b3-5e98-4d16-8158-afba6fe42281-tmp-dir\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:05:58.374644 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.374508    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxhpm\" (UniqueName: \"kubernetes.io/projected/ce07e9b3-5e98-4d16-8158-afba6fe42281-kube-api-access-wxhpm\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:05:58.374686 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.374669    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc2g2\" (UniqueName: \"kubernetes.io/projected/7411c16b-533b-4152-bbe6-52262afa37da-kube-api-access-wc2g2\") pod \"ingress-canary-xtj4p\" (UID: \"7411c16b-533b-4152-bbe6-52262afa37da\") " pod="openshift-ingress-canary/ingress-canary-xtj4p"
Oct 30 19:05:58.374720 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.374690    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce07e9b3-5e98-4d16-8158-afba6fe42281-metrics-tls\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:05:58.378263 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.378244    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7411c16b-533b-4152-bbe6-52262afa37da-cert\") pod \"ingress-canary-xtj4p\" (UID: \"7411c16b-533b-4152-bbe6-52262afa37da\") " pod="openshift-ingress-canary/ingress-canary-xtj4p"
Oct 30 19:05:58.387044 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.387015    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc2g2\" (UniqueName: \"kubernetes.io/projected/7411c16b-533b-4152-bbe6-52262afa37da-kube-api-access-wc2g2\") pod \"ingress-canary-xtj4p\" (UID: \"7411c16b-533b-4152-bbe6-52262afa37da\") " pod="openshift-ingress-canary/ingress-canary-xtj4p"
Oct 30 19:05:58.475905 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.475881    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce07e9b3-5e98-4d16-8158-afba6fe42281-metrics-tls\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:05:58.475977 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.475910    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce07e9b3-5e98-4d16-8158-afba6fe42281-config-volume\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:05:58.475977 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.475925    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce07e9b3-5e98-4d16-8158-afba6fe42281-tmp-dir\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:05:58.475977 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.475943    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxhpm\" (UniqueName: \"kubernetes.io/projected/ce07e9b3-5e98-4d16-8158-afba6fe42281-kube-api-access-wxhpm\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:05:58.476071 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:58.476039    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 19:05:58.476110 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:58.476100    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce07e9b3-5e98-4d16-8158-afba6fe42281-metrics-tls podName:ce07e9b3-5e98-4d16-8158-afba6fe42281 nodeName:}" failed. No retries permitted until 2026-10-30 19:05:58.976083215 +0000 UTC m=+31.700347923 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce07e9b3-5e98-4d16-8158-afba6fe42281-metrics-tls") pod "dns-default-v86zw" (UID: "ce07e9b3-5e98-4d16-8158-afba6fe42281") : secret "dns-default-metrics-tls" not found
Oct 30 19:05:58.476305 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.476285    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce07e9b3-5e98-4d16-8158-afba6fe42281-tmp-dir\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:05:58.476491 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.476476    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce07e9b3-5e98-4d16-8158-afba6fe42281-config-volume\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:05:58.488537 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.488514    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxhpm\" (UniqueName: \"kubernetes.io/projected/ce07e9b3-5e98-4d16-8158-afba6fe42281-kube-api-access-wxhpm\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:05:58.537572 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.537547    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xtj4p"
Oct 30 19:05:58.701386 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.701362    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:05:58.701520 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.701386    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:05:58.701593 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.701581    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:05:58.705943 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.705890    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 19:05:58.705943 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.705904    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 19:05:58.706102 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.706004    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fljvz\""
Oct 30 19:05:58.706102 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.706018    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 19:05:58.706102 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.706023    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-tc7v2\""
Oct 30 19:05:58.706102 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.706015    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-89jvk\""
Oct 30 19:05:58.706102 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.706062    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 19:05:58.877995 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.877964    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xtj4p"]
Oct 30 19:05:58.882230 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:05:58.882205    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7411c16b_533b_4152_bbe6_52262afa37da.slice/crio-1c1ff350bbc22b21f236f6650a0e079cf70a79cac5f1f0878f7ac1a518b4d21e WatchSource:0}: Error finding container 1c1ff350bbc22b21f236f6650a0e079cf70a79cac5f1f0878f7ac1a518b4d21e: Status 404 returned error can't find the container with id 1c1ff350bbc22b21f236f6650a0e079cf70a79cac5f1f0878f7ac1a518b4d21e
Oct 30 19:05:58.936350 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.936281    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xtj4p" event={"ID":"7411c16b-533b-4152-bbe6-52262afa37da","Type":"ContainerStarted","Data":"1c1ff350bbc22b21f236f6650a0e079cf70a79cac5f1f0878f7ac1a518b4d21e"}
Oct 30 19:05:58.980427 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:58.980401    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce07e9b3-5e98-4d16-8158-afba6fe42281-metrics-tls\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:05:58.980541 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:58.980526    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 19:05:58.980594 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:58.980586    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce07e9b3-5e98-4d16-8158-afba6fe42281-metrics-tls podName:ce07e9b3-5e98-4d16-8158-afba6fe42281 nodeName:}" failed. No retries permitted until 2026-10-30 19:05:59.980563265 +0000 UTC m=+32.704827960 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce07e9b3-5e98-4d16-8158-afba6fe42281-metrics-tls") pod "dns-default-v86zw" (UID: "ce07e9b3-5e98-4d16-8158-afba6fe42281") : secret "dns-default-metrics-tls" not found
Oct 30 19:05:59.843899 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.843864    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-75z8h-spdtl"]
Oct 30 19:05:59.868190 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.868152    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fht7f"]
Oct 30 19:05:59.868347 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.868259    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:05:59.872082 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.872059    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 19:05:59.884420 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.884400    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-75z8h-spdtl"]
Oct 30 19:05:59.884522 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.884435    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fht7f"]
Oct 30 19:05:59.884522 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.884486    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:05:59.965325 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.965295    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 19:05:59.965325 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.965295    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 19:05:59.965565 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.965392    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 19:05:59.965565 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.965406    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 19:05:59.965565 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.965544    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fsc2n\""
Oct 30 19:05:59.965729 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.965628    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-stgll\""
Oct 30 19:05:59.970574 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.970553    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 19:05:59.985785 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.985763    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqgn5\" (UniqueName: \"kubernetes.io/projected/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-kube-api-access-wqgn5\") pod \"periodic-gathering-75z8h-spdtl\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") " pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:05:59.985902 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.985792    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-service-ca-bundle\") pod \"periodic-gathering-75z8h-spdtl\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") " pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:05:59.985956 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.985917    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce07e9b3-5e98-4d16-8158-afba6fe42281-metrics-tls\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:05:59.985956 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.985944    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-archives-path\") pod \"periodic-gathering-75z8h-spdtl\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") " pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:05:59.986039 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.985961    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-trusted-ca-bundle\") pod \"periodic-gathering-75z8h-spdtl\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") " pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:05:59.986039 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.985993    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c4b9f9bd-6097-4524-9c86-44c7f484f278-crio-socket\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:05:59.986039 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.986027    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:05:59.986133 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:59.986048    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 19:05:59.986133 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.986065    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4b9f9bd-6097-4524-9c86-44c7f484f278-data-volume\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:05:59.986229 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:05:59.986155    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce07e9b3-5e98-4d16-8158-afba6fe42281-metrics-tls podName:ce07e9b3-5e98-4d16-8158-afba6fe42281 nodeName:}" failed. No retries permitted until 2026-10-30 19:06:01.98613221 +0000 UTC m=+34.710396920 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce07e9b3-5e98-4d16-8158-afba6fe42281-metrics-tls") pod "dns-default-v86zw" (UID: "ce07e9b3-5e98-4d16-8158-afba6fe42281") : secret "dns-default-metrics-tls" not found
Oct 30 19:05:59.986229 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.986214    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:05:59.986299 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:05:59.986235    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jmzz\" (UniqueName: \"kubernetes.io/projected/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-api-access-6jmzz\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:00.087492 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.087459    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqgn5\" (UniqueName: \"kubernetes.io/projected/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-kube-api-access-wqgn5\") pod \"periodic-gathering-75z8h-spdtl\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") " pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:06:00.087492 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.087492    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-service-ca-bundle\") pod \"periodic-gathering-75z8h-spdtl\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") " pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:06:00.087720 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.087559    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-archives-path\") pod \"periodic-gathering-75z8h-spdtl\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") " pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:06:00.087720 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.087575    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-trusted-ca-bundle\") pod \"periodic-gathering-75z8h-spdtl\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") " pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:06:00.087720 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.087601    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c4b9f9bd-6097-4524-9c86-44c7f484f278-crio-socket\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:00.087720 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.087618    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:00.087720 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.087633    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4b9f9bd-6097-4524-9c86-44c7f484f278-data-volume\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:00.087720 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.087657    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:00.087720 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.087683    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jmzz\" (UniqueName: \"kubernetes.io/projected/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-api-access-6jmzz\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:00.087963 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.087930    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c4b9f9bd-6097-4524-9c86-44c7f484f278-crio-socket\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:00.087963 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:00.087944    2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 19:06:00.088145 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:00.088008    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-rbac-proxy-cm podName:c4b9f9bd-6097-4524-9c86-44c7f484f278 nodeName:}" failed. No retries permitted until 2026-10-30 19:06:00.587989595 +0000 UTC m=+33.312254314 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fht7f" (UID: "c4b9f9bd-6097-4524-9c86-44c7f484f278") : configmap "kube-rbac-proxy" not found
Oct 30 19:06:00.088234 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.088158    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4b9f9bd-6097-4524-9c86-44c7f484f278-data-volume\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:00.088234 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.088211    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-service-ca-bundle\") pod \"periodic-gathering-75z8h-spdtl\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") " pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:06:00.088341 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:00.088245    2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 19:06:00.088341 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:00.088281    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls podName:c4b9f9bd-6097-4524-9c86-44c7f484f278 nodeName:}" failed. No retries permitted until 2026-10-30 19:06:00.588270488 +0000 UTC m=+33.312535208 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fht7f" (UID: "c4b9f9bd-6097-4524-9c86-44c7f484f278") : secret "insights-runtime-extractor-tls" not found
Oct 30 19:06:00.088471 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.088450    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-archives-path\") pod \"periodic-gathering-75z8h-spdtl\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") " pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:06:00.089305 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.089287    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-trusted-ca-bundle\") pod \"periodic-gathering-75z8h-spdtl\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") " pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:06:00.101083 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.101023    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jmzz\" (UniqueName: \"kubernetes.io/projected/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-api-access-6jmzz\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:00.101368 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.101337    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqgn5\" (UniqueName: \"kubernetes.io/projected/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-kube-api-access-wqgn5\") pod \"periodic-gathering-75z8h-spdtl\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") " pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:06:00.191519 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.191480    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:06:00.355616 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.355546    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-svnhq"]
Oct 30 19:06:00.387918 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.387882    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-svnhq"]
Oct 30 19:06:00.388085 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.388006    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-svnhq"
Oct 30 19:06:00.484924 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.484888    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 19:06:00.484924 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.484921    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 19:06:00.485223 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.485205    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-6qlhx\""
Oct 30 19:06:00.490423 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.490399    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw4hs\" (UniqueName: \"kubernetes.io/projected/5fdb2ea2-a56c-46f0-b155-468303552965-kube-api-access-qw4hs\") pod \"migrator-66df5d496b-svnhq\" (UID: \"5fdb2ea2-a56c-46f0-b155-468303552965\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-svnhq"
Oct 30 19:06:00.498155 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.498133    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-75z8h-spdtl"]
Oct 30 19:06:00.501313 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:06:00.501293    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ac4e9e2_d815_4b3b_86cd_db537f984e6c.slice/crio-e00ec3b69fb49a9276c87d5dc5a8dd5226a2db07f48ab9f9d4781dcf0c682ccc WatchSource:0}: Error finding container e00ec3b69fb49a9276c87d5dc5a8dd5226a2db07f48ab9f9d4781dcf0c682ccc: Status 404 returned error can't find the container with id e00ec3b69fb49a9276c87d5dc5a8dd5226a2db07f48ab9f9d4781dcf0c682ccc
Oct 30 19:06:00.591023 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.590997    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:00.591126 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.591030    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:00.591126 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.591050    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw4hs\" (UniqueName: \"kubernetes.io/projected/5fdb2ea2-a56c-46f0-b155-468303552965-kube-api-access-qw4hs\") pod \"migrator-66df5d496b-svnhq\" (UID: \"5fdb2ea2-a56c-46f0-b155-468303552965\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-svnhq"
Oct 30 19:06:00.591126 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:00.591110    2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 19:06:00.591246 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:00.591206    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-rbac-proxy-cm podName:c4b9f9bd-6097-4524-9c86-44c7f484f278 nodeName:}" failed. No retries permitted until 2026-10-30 19:06:01.591153759 +0000 UTC m=+34.315418454 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fht7f" (UID: "c4b9f9bd-6097-4524-9c86-44c7f484f278") : configmap "kube-rbac-proxy" not found
Oct 30 19:06:00.591246 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:00.591214    2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 19:06:00.591320 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:00.591278    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls podName:c4b9f9bd-6097-4524-9c86-44c7f484f278 nodeName:}" failed. No retries permitted until 2026-10-30 19:06:01.591262967 +0000 UTC m=+34.315527667 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fht7f" (UID: "c4b9f9bd-6097-4524-9c86-44c7f484f278") : secret "insights-runtime-extractor-tls" not found
Oct 30 19:06:00.604879 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.604850    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw4hs\" (UniqueName: \"kubernetes.io/projected/5fdb2ea2-a56c-46f0-b155-468303552965-kube-api-access-qw4hs\") pod \"migrator-66df5d496b-svnhq\" (UID: \"5fdb2ea2-a56c-46f0-b155-468303552965\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-svnhq"
Oct 30 19:06:00.703755 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.703670    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-svnhq"
Oct 30 19:06:00.857558 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.857519    2586 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-5nrmq"]
Oct 30 19:06:00.869616 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.869587    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-5nrmq"
Oct 30 19:06:00.871880 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.871859    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-5nrmq"]
Oct 30 19:06:00.877438 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.877403    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 30 19:06:00.877438 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.877409    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 30 19:06:00.877612 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.877466    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 30 19:06:00.877612 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.877516    2586 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 30 19:06:00.877612 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.877531    2586 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-9klsx\""
Oct 30 19:06:00.941526 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.941492    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-75z8h-spdtl" event={"ID":"2ac4e9e2-d815-4b3b-86cd-db537f984e6c","Type":"ContainerStarted","Data":"e00ec3b69fb49a9276c87d5dc5a8dd5226a2db07f48ab9f9d4781dcf0c682ccc"}
Oct 30 19:06:00.994002 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.993912    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d36c6f3a-c76b-4010-898b-9cd567933c5a-signing-cabundle\") pod \"service-ca-5fffc97bdf-5nrmq\" (UID: \"d36c6f3a-c76b-4010-898b-9cd567933c5a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-5nrmq"
Oct 30 19:06:00.994163 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.994021    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxkwv\" (UniqueName: \"kubernetes.io/projected/d36c6f3a-c76b-4010-898b-9cd567933c5a-kube-api-access-vxkwv\") pod \"service-ca-5fffc97bdf-5nrmq\" (UID: \"d36c6f3a-c76b-4010-898b-9cd567933c5a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-5nrmq"
Oct 30 19:06:00.994163 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.994138    2586 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d36c6f3a-c76b-4010-898b-9cd567933c5a-signing-key\") pod \"service-ca-5fffc97bdf-5nrmq\" (UID: \"d36c6f3a-c76b-4010-898b-9cd567933c5a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-5nrmq"
Oct 30 19:06:00.997360 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:00.997337    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-svnhq"]
Oct 30 19:06:01.000972 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:06:01.000948    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fdb2ea2_a56c_46f0_b155_468303552965.slice/crio-3fe2c2a4e97d682721b15997a4fa394a0bcb7da534ba776f54e789550e53d96b WatchSource:0}: Error finding container 3fe2c2a4e97d682721b15997a4fa394a0bcb7da534ba776f54e789550e53d96b: Status 404 returned error can't find the container with id 3fe2c2a4e97d682721b15997a4fa394a0bcb7da534ba776f54e789550e53d96b
Oct 30 19:06:01.095712 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:01.095665    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d36c6f3a-c76b-4010-898b-9cd567933c5a-signing-key\") pod \"service-ca-5fffc97bdf-5nrmq\" (UID: \"d36c6f3a-c76b-4010-898b-9cd567933c5a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-5nrmq"
Oct 30 19:06:01.095878 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:01.095846    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d36c6f3a-c76b-4010-898b-9cd567933c5a-signing-cabundle\") pod \"service-ca-5fffc97bdf-5nrmq\" (UID: \"d36c6f3a-c76b-4010-898b-9cd567933c5a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-5nrmq"
Oct 30 19:06:01.095939 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:01.095884    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxkwv\" (UniqueName: \"kubernetes.io/projected/d36c6f3a-c76b-4010-898b-9cd567933c5a-kube-api-access-vxkwv\") pod \"service-ca-5fffc97bdf-5nrmq\" (UID: \"d36c6f3a-c76b-4010-898b-9cd567933c5a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-5nrmq"
Oct 30 19:06:01.096624 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:01.096601    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d36c6f3a-c76b-4010-898b-9cd567933c5a-signing-cabundle\") pod \"service-ca-5fffc97bdf-5nrmq\" (UID: \"d36c6f3a-c76b-4010-898b-9cd567933c5a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-5nrmq"
Oct 30 19:06:01.098427 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:01.098404    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d36c6f3a-c76b-4010-898b-9cd567933c5a-signing-key\") pod \"service-ca-5fffc97bdf-5nrmq\" (UID: \"d36c6f3a-c76b-4010-898b-9cd567933c5a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-5nrmq"
Oct 30 19:06:01.109777 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:01.109752    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxkwv\" (UniqueName: \"kubernetes.io/projected/d36c6f3a-c76b-4010-898b-9cd567933c5a-kube-api-access-vxkwv\") pod \"service-ca-5fffc97bdf-5nrmq\" (UID: \"d36c6f3a-c76b-4010-898b-9cd567933c5a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-5nrmq"
Oct 30 19:06:01.203500 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:01.203467    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-5nrmq"
Oct 30 19:06:01.398896 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:01.398859    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs\") pod \"network-metrics-daemon-xmkt2\" (UID: \"ed4e7b61-d816-49fc-b4d9-36236a716227\") " pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:06:01.399068 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:01.398999    2586 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Oct 30 19:06:01.399068 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:01.399059    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs podName:ed4e7b61-d816-49fc-b4d9-36236a716227 nodeName:}" failed. No retries permitted until 2026-10-30 19:06:33.399041945 +0000 UTC m=+66.123306640 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs") pod "network-metrics-daemon-xmkt2" (UID: "ed4e7b61-d816-49fc-b4d9-36236a716227") : secret "metrics-daemon-secret" not found
Oct 30 19:06:01.500360 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:01.500289    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:06:01.502719 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:01.502689    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:06:01.523609 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:01.523578    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-5nrmq"]
Oct 30 19:06:01.528245 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:06:01.528216    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd36c6f3a_c76b_4010_898b_9cd567933c5a.slice/crio-3421b7bfa5bcba71023eedf52a8df24102943c22620eb2e3400eaf57f5846207 WatchSource:0}: Error finding container 3421b7bfa5bcba71023eedf52a8df24102943c22620eb2e3400eaf57f5846207: Status 404 returned error can't find the container with id 3421b7bfa5bcba71023eedf52a8df24102943c22620eb2e3400eaf57f5846207
Oct 30 19:06:01.601064 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:01.601026    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:01.601211 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:01.601066    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:01.601211 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:01.601122    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pq2q\" (UniqueName: \"kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q\") pod \"network-check-target-z2dbh\" (UID: \"c1cc12f1-be52-4a9f-a8a2-53fe3c493431\") " pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:06:01.601211 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:01.601183    2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 19:06:01.601368 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:01.601253    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-rbac-proxy-cm podName:c4b9f9bd-6097-4524-9c86-44c7f484f278 nodeName:}" failed. No retries permitted until 2026-10-30 19:06:03.601233749 +0000 UTC m=+36.325498450 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fht7f" (UID: "c4b9f9bd-6097-4524-9c86-44c7f484f278") : configmap "kube-rbac-proxy" not found
Oct 30 19:06:01.601368 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:01.601269    2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 19:06:01.601368 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:01.601329    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls podName:c4b9f9bd-6097-4524-9c86-44c7f484f278 nodeName:}" failed. No retries permitted until 2026-10-30 19:06:03.601311558 +0000 UTC m=+36.325576277 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fht7f" (UID: "c4b9f9bd-6097-4524-9c86-44c7f484f278") : secret "insights-runtime-extractor-tls" not found
Oct 30 19:06:01.604707 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:01.604682    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pq2q\" (UniqueName: \"kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q\") pod \"network-check-target-z2dbh\" (UID: \"c1cc12f1-be52-4a9f-a8a2-53fe3c493431\") " pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:06:01.723548 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:01.723513    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:06:01.727649 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:01.727619    2586 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:06:01.946138 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:01.946110    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-svnhq" event={"ID":"5fdb2ea2-a56c-46f0-b155-468303552965","Type":"ContainerStarted","Data":"3fe2c2a4e97d682721b15997a4fa394a0bcb7da534ba776f54e789550e53d96b"}
Oct 30 19:06:01.947234 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:01.947209    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-5nrmq" event={"ID":"d36c6f3a-c76b-4010-898b-9cd567933c5a","Type":"ContainerStarted","Data":"3421b7bfa5bcba71023eedf52a8df24102943c22620eb2e3400eaf57f5846207"}
Oct 30 19:06:02.004771 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:02.004700    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce07e9b3-5e98-4d16-8158-afba6fe42281-metrics-tls\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:06:02.004890 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:02.004833    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 19:06:02.004890 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:02.004887    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce07e9b3-5e98-4d16-8158-afba6fe42281-metrics-tls podName:ce07e9b3-5e98-4d16-8158-afba6fe42281 nodeName:}" failed. No retries permitted until 2026-10-30 19:06:06.004869641 +0000 UTC m=+38.729134339 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce07e9b3-5e98-4d16-8158-afba6fe42281-metrics-tls") pod "dns-default-v86zw" (UID: "ce07e9b3-5e98-4d16-8158-afba6fe42281") : secret "dns-default-metrics-tls" not found
Oct 30 19:06:02.169343 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:02.169284    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z2dbh"]
Oct 30 19:06:02.169402 ip-10-0-137-200 kubenswrapper[2586]: W1030 19:06:02.169348    2586 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1cc12f1_be52_4a9f_a8a2_53fe3c493431.slice/crio-2847ba93a9d1fec6ee5a390815f406cee1525845709e5b52dea049ee3a0ab759 WatchSource:0}: Error finding container 2847ba93a9d1fec6ee5a390815f406cee1525845709e5b52dea049ee3a0ab759: Status 404 returned error can't find the container with id 2847ba93a9d1fec6ee5a390815f406cee1525845709e5b52dea049ee3a0ab759
Oct 30 19:06:02.170496 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:02.170473    2586 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7q5vl"]
Oct 30 19:06:02.951438 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:02.951406    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-svnhq" event={"ID":"5fdb2ea2-a56c-46f0-b155-468303552965","Type":"ContainerStarted","Data":"db4d8544fbb8419ef4e1fb880e2eef9fe28bf33c830faaf87431a6cc9ca65dad"}
Oct 30 19:06:02.952955 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:02.952920    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7q5vl" event={"ID":"ba82c577-b89d-44f7-87e4-2d00d1e2a5de","Type":"ContainerStarted","Data":"a695b8224522047769b03fbca685bcd7ba26b1b00fc365e370099af9cc14b4d3"}
Oct 30 19:06:02.954952 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:02.954501    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xtj4p" event={"ID":"7411c16b-533b-4152-bbe6-52262afa37da","Type":"ContainerStarted","Data":"b27b4892a663b79f069e3b642062ce8d571689bea6a6c77bd7c62ad848da239d"}
Oct 30 19:06:02.955864 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:02.955822    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z2dbh" event={"ID":"c1cc12f1-be52-4a9f-a8a2-53fe3c493431","Type":"ContainerStarted","Data":"2847ba93a9d1fec6ee5a390815f406cee1525845709e5b52dea049ee3a0ab759"}
Oct 30 19:06:02.971206 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:02.970616    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-svnhq" podStartSLOduration=1.229364756 podStartE2EDuration="2.970599579s" totalImagesPullingTime="1.741234823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:02.968997935 +0000 UTC m=+35.693262652" watchObservedRunningTime="2026-10-30 19:06:02.970599579 +0000 UTC m=+35.694864297"
Oct 30 19:06:02.987998 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:02.987954    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xtj4p" podStartSLOduration=1.98770539 podStartE2EDuration="4.987939255s" totalImagesPullingTime="3.000233865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:02.987199232 +0000 UTC m=+35.711463952" watchObservedRunningTime="2026-10-30 19:06:02.987939255 +0000 UTC m=+35.712203975"
Oct 30 19:06:03.622162 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:03.622123    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:03.622365 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:03.622199    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:03.622365 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:03.622261    2586 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Oct 30 19:06:03.622365 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:03.622336    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-rbac-proxy-cm podName:c4b9f9bd-6097-4524-9c86-44c7f484f278 nodeName:}" failed. No retries permitted until 2026-10-30 19:06:07.622315585 +0000 UTC m=+40.346580297 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fht7f" (UID: "c4b9f9bd-6097-4524-9c86-44c7f484f278") : configmap "kube-rbac-proxy" not found
Oct 30 19:06:03.622561 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:03.622380    2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 19:06:03.622561 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:03.622434    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls podName:c4b9f9bd-6097-4524-9c86-44c7f484f278 nodeName:}" failed. No retries permitted until 2026-10-30 19:06:07.622418054 +0000 UTC m=+40.346682753 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fht7f" (UID: "c4b9f9bd-6097-4524-9c86-44c7f484f278") : secret "insights-runtime-extractor-tls" not found
Oct 30 19:06:03.961207 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:03.961106    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-75z8h-spdtl" event={"ID":"2ac4e9e2-d815-4b3b-86cd-db537f984e6c","Type":"ContainerStarted","Data":"b44637b0d0eef1f1754b5656466bbc08de9d3d662fadcf883d9d596602a91542"}
Oct 30 19:06:03.981767 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:03.981712    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-75z8h-spdtl" podStartSLOduration=2.417075267 podStartE2EDuration="4.98169272s" totalImagesPullingTime="2.564617453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:03.979646997 +0000 UTC m=+36.703911716" watchObservedRunningTime="2026-10-30 19:06:03.98169272 +0000 UTC m=+36.705957439"
Oct 30 19:06:04.966647 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:04.966611    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-5nrmq" event={"ID":"d36c6f3a-c76b-4010-898b-9cd567933c5a","Type":"ContainerStarted","Data":"90e3b69c803028e3990bb9f1fcd1b14bebe07cdc2f7c1935f66e1133d42446d4"}
Oct 30 19:06:04.987719 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:04.987665    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-5nrmq" podStartSLOduration=2.061430718 podStartE2EDuration="4.987651774s" totalImagesPullingTime="2.926221056s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:04.985736222 +0000 UTC m=+37.710000940" watchObservedRunningTime="2026-10-30 19:06:04.987651774 +0000 UTC m=+37.711916488"
Oct 30 19:06:06.045486 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:06.045456    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce07e9b3-5e98-4d16-8158-afba6fe42281-metrics-tls\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:06:06.046102 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:06.045624    2586 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Oct 30 19:06:06.046102 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:06.045689    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce07e9b3-5e98-4d16-8158-afba6fe42281-metrics-tls podName:ce07e9b3-5e98-4d16-8158-afba6fe42281 nodeName:}" failed. No retries permitted until 2026-10-30 19:06:14.045668915 +0000 UTC m=+46.769933627 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce07e9b3-5e98-4d16-8158-afba6fe42281-metrics-tls") pod "dns-default-v86zw" (UID: "ce07e9b3-5e98-4d16-8158-afba6fe42281") : secret "dns-default-metrics-tls" not found
Oct 30 19:06:06.471882 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:06.471858    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8jrs8_94709da2-704a-421c-aa2f-1844db5bec96/dns-node-resolver/0.log"
Oct 30 19:06:06.975197 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:06.973544    2586 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z2dbh" event={"ID":"c1cc12f1-be52-4a9f-a8a2-53fe3c493431","Type":"ContainerStarted","Data":"9a470d09a289e846502a266c933e728a6a0669385d279468e08e8f0e086ac8c3"}
Oct 30 19:06:06.975197 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:06.973660    2586 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:06:07.004941 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:07.004883    2586 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z2dbh" podStartSLOduration=35.551183208 podStartE2EDuration="40.00484079s" totalImagesPullingTime="4.453657582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:07.002730137 +0000 UTC m=+39.726994855" watchObservedRunningTime="2026-10-30 19:06:07.00484079 +0000 UTC m=+39.729105511"
Oct 30 19:06:07.279878 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:07.279807    2586 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2z2fd_838f8f5e-ac73-4280-8ed3-173a51c42b9e/node-ca/0.log"
Oct 30 19:06:07.659521 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:07.659487    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:07.659666 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:07.659535    2586 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:07.659719 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:07.659682    2586 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 19:06:07.659753 ip-10-0-137-200 kubenswrapper[2586]: E1030 19:06:07.659740    2586 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls podName:c4b9f9bd-6097-4524-9c86-44c7f484f278 nodeName:}" failed. No retries permitted until 2026-10-30 19:06:15.659723305 +0000 UTC m=+48.383988005 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fht7f" (UID: "c4b9f9bd-6097-4524-9c86-44c7f484f278") : secret "insights-runtime-extractor-tls" not found
Oct 30 19:06:07.659983 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:07.659965    2586 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:08.192897 ip-10-0-137-200 kubenswrapper[2586]: I1030 19:06:08.192778    2586 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 19:06:08.192944 ip-10-0-137-200 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 19:06:08.211568 ip-10-0-137-200 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 19:06:08.211749 ip-10-0-137-200 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 19:06:08.211797 ip-10-0-137-200 systemd[1]: kubelet.service: Consumed 2.305s CPU time.
Oct 30 19:06:08.229676 ip-10-0-137-200 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 19:06:08.303704 ip-10-0-137-200 kubenswrapper[4524]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:06:08.303704 ip-10-0-137-200 kubenswrapper[4524]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 19:06:08.303704 ip-10-0-137-200 kubenswrapper[4524]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:06:08.303704 ip-10-0-137-200 kubenswrapper[4524]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 19:06:08.305745 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305667    4524 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:06:08.305745 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305744    4524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:06:08.305820 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305748    4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:06:08.305820 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305752    4524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:06:08.305820 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305756    4524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:06:08.305820 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305759    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:06:08.305820 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305762    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:06:08.305820 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305764    4524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:06:08.305820 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305767    4524 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:06:08.305820 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305770    4524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:06:08.305820 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305773    4524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:06:08.305820 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305776    4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:06:08.305820 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305779    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:06:08.305820 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305781    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:06:08.305820 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305790    4524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:06:08.305820 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305793    4524 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:06:08.305820 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305796    4524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:06:08.305820 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305798    4524 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:06:08.305820 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305801    4524 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:06:08.305820 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305804    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:06:08.305820 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305806    4524 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:06:08.305820 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305809    4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:06:08.306333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305812    4524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:06:08.306333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305814    4524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:06:08.306333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305817    4524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:06:08.306333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305820    4524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:06:08.306333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305822    4524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:06:08.306333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305825    4524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:06:08.306333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305828    4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:06:08.306333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305831    4524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:06:08.306333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305834    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:06:08.306333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305836    4524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:06:08.306333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305839    4524 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:06:08.306333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305841    4524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:06:08.306333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305844    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:06:08.306333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305846    4524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:06:08.306333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305849    4524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:06:08.306333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305851    4524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:06:08.306333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305854    4524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:06:08.306333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305856    4524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:06:08.306333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305859    4524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:06:08.306813 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305861    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:06:08.306813 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305864    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:06:08.306813 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305866    4524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:06:08.306813 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305869    4524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:06:08.306813 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305871    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:06:08.306813 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305874    4524 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:06:08.306813 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305877    4524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:06:08.306813 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305879    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:06:08.306813 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305882    4524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:06:08.306813 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305885    4524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:06:08.306813 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305888    4524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:06:08.306813 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305891    4524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:06:08.306813 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305893    4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:06:08.306813 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305895    4524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:06:08.306813 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305898    4524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:06:08.306813 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305901    4524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:06:08.306813 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305903    4524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:06:08.306813 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305906    4524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:06:08.306813 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305908    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:06:08.306813 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305911    4524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:06:08.307333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305913    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:06:08.307333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305916    4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:06:08.307333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305918    4524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:06:08.307333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305921    4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:06:08.307333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305923    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:06:08.307333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305926    4524 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:06:08.307333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305928    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:06:08.307333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305933    4524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:06:08.307333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305937    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:06:08.307333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305940    4524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:06:08.307333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305942    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:06:08.307333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305945    4524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:06:08.307333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305948    4524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:06:08.307333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305951    4524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:06:08.307333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305953    4524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:06:08.307333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305956    4524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:06:08.307333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305958    4524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:06:08.307333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305961    4524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:06:08.307333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305963    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:06:08.307851 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305966    4524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:06:08.307851 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305969    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:06:08.307851 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305972    4524 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:06:08.307851 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305974    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:06:08.307851 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305977    4524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:06:08.307851 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305980    4524 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:06:08.307851 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305982    4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:06:08.307851 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305986    4524 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:06:08.307851 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305990    4524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:06:08.307851 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305994    4524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:06:08.307851 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.305997    4524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:06:08.307851 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306001    4524 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:06:08.307851 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306003    4524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:06:08.307851 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306006    4524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:06:08.307851 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306009    4524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:06:08.307851 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306011    4524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:06:08.307851 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306014    4524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:06:08.307851 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306016    4524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:06:08.307851 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306019    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:06:08.308367 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306022    4524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:06:08.308367 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306024    4524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:06:08.308367 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306028    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:06:08.308367 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306030    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:06:08.308367 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306033    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:06:08.308367 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306036    4524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:06:08.308367 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306039    4524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:06:08.308367 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306804    4524 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:06:08.308367 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306811    4524 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:06:08.308367 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306814    4524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:06:08.308367 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306817    4524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:06:08.308367 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306820    4524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:06:08.308367 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306823    4524 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:06:08.308367 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306826    4524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:06:08.308367 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306832    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:06:08.308367 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306835    4524 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:06:08.308367 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306838    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:06:08.308367 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306842    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:06:08.308367 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306844    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:06:08.308367 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306847    4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:06:08.308890 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306850    4524 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:06:08.308890 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306853    4524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:06:08.308890 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306856    4524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:06:08.308890 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306858    4524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:06:08.308890 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306861    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:06:08.308890 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306864    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:06:08.308890 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306866    4524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:06:08.308890 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306869    4524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:06:08.308890 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306872    4524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:06:08.308890 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306874    4524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:06:08.308890 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306877    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:06:08.308890 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306880    4524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:06:08.308890 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306883    4524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:06:08.308890 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306885    4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:06:08.308890 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306888    4524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:06:08.308890 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306890    4524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:06:08.308890 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306895    4524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:06:08.308890 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306899    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:06:08.308890 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306903    4524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:06:08.309378 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306906    4524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:06:08.309378 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306909    4524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:06:08.309378 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306911    4524 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:06:08.309378 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306914    4524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:06:08.309378 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306917    4524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:06:08.309378 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306919    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:06:08.309378 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306922    4524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:06:08.309378 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306925    4524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:06:08.309378 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306928    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:06:08.309378 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306931    4524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:06:08.309378 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306934    4524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:06:08.309378 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306936    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:06:08.309378 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306939    4524 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:06:08.309378 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306944    4524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:06:08.309378 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306947    4524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:06:08.309378 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306950    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:06:08.309378 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306953    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:06:08.309378 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306956    4524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:06:08.309378 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306959    4524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:06:08.309844 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306961    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:06:08.309844 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306964    4524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:06:08.309844 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306967    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:06:08.309844 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306969    4524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:06:08.309844 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306972    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:06:08.309844 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306974    4524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:06:08.309844 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306977    4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:06:08.309844 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306980    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:06:08.309844 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306982    4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:06:08.309844 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306985    4524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:06:08.309844 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306987    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:06:08.309844 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306990    4524 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:06:08.309844 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306993    4524 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:06:08.309844 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306995    4524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:06:08.309844 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.306998    4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:06:08.309844 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307000    4524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:06:08.309844 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307003    4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:06:08.309844 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307005    4524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:06:08.309844 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307008    4524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:06:08.310326 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307011    4524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:06:08.310326 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307014    4524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:06:08.310326 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307017    4524 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:06:08.310326 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307020    4524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:06:08.310326 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307023    4524 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:06:08.310326 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307025    4524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:06:08.310326 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307029    4524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:06:08.310326 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307031    4524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:06:08.310326 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307034    4524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:06:08.310326 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307036    4524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:06:08.310326 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307039    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:06:08.310326 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307041    4524 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:06:08.310326 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307044    4524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:06:08.310326 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307046    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:06:08.310326 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307049    4524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:06:08.310326 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307051    4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:06:08.310326 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307054    4524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:06:08.310326 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307056    4524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:06:08.310326 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307059    4524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:06:08.310326 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307061    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:06:08.310811 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307064    4524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:06:08.310811 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307067    4524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:06:08.310811 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307070    4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:06:08.310811 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307072    4524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:06:08.310811 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307074    4524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:06:08.310811 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307077    4524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:06:08.310811 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307080    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:06:08.310811 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307082    4524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:06:08.310811 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307085    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:06:08.310811 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307088    4524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:06:08.310811 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307090    4524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:06:08.310811 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307093    4524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:06:08.310811 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307096    4524 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:06:08.310811 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307099    4524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:06:08.310811 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307101    4524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:06:08.310811 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307105    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:06:08.310811 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307602    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:06:08.310811 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307608    4524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:06:08.310811 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307611    4524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:06:08.310811 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307614    4524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:06:08.311317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307616    4524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:06:08.311317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307619    4524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:06:08.311317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307622    4524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:06:08.311317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307624    4524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:06:08.311317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307627    4524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:06:08.311317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307629    4524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:06:08.311317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307632    4524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:06:08.311317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307635    4524 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:06:08.311317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307638    4524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:06:08.311317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307641    4524 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:06:08.311317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307644    4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:06:08.311317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307646    4524 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:06:08.311317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307649    4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:06:08.311317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307652    4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:06:08.311317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307654    4524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:06:08.311317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307657    4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:06:08.311317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307660    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:06:08.311317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307662    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:06:08.311317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307665    4524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:06:08.311317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307668    4524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:06:08.311900 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307670    4524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:06:08.311900 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307683    4524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:06:08.311900 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307688    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:06:08.311900 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307694    4524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:06:08.311900 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307697    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:06:08.311900 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307700    4524 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:06:08.311900 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307703    4524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:06:08.311900 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307705    4524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:06:08.311900 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307709    4524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:06:08.311900 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307711    4524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:06:08.311900 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307714    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:06:08.311900 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307716    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:06:08.311900 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307719    4524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:06:08.311900 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307722    4524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:06:08.311900 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307724    4524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:06:08.311900 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307727    4524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:06:08.311900 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307730    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:06:08.311900 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307732    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:06:08.311900 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307736    4524 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:06:08.312666 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307738    4524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:06:08.312666 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307743    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:06:08.312666 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307746    4524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:06:08.312666 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307748    4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:06:08.312666 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307751    4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:06:08.312666 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307754    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:06:08.312666 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307756    4524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:06:08.312666 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307759    4524 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:06:08.312666 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307761    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:06:08.312666 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307764    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:06:08.312666 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307766    4524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:06:08.312666 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307769    4524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:06:08.312666 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307771    4524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:06:08.312666 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307774    4524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:06:08.312666 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307776    4524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:06:08.312666 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307779    4524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:06:08.312666 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307783    4524 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:06:08.312666 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307786    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:06:08.312666 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307788    4524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:06:08.313333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307791    4524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:06:08.313333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307793    4524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:06:08.313333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307796    4524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:06:08.313333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307803    4524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:06:08.313333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307807    4524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:06:08.313333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307810    4524 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:06:08.313333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307812    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:06:08.313333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307815    4524 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:06:08.313333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307818    4524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:06:08.313333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307820    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:06:08.313333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307823    4524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:06:08.313333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307826    4524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:06:08.313333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307829    4524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:06:08.313333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307832    4524 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:06:08.313333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307834    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:06:08.313333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307837    4524 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:06:08.313333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307840    4524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:06:08.313333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307842    4524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:06:08.313333 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307845    4524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:06:08.313975 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307847    4524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:06:08.313975 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307850    4524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:06:08.313975 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307852    4524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:06:08.313975 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307855    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:06:08.313975 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307858    4524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:06:08.313975 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307860    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:06:08.313975 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307863    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:06:08.313975 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307865    4524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:06:08.313975 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307868    4524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:06:08.313975 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307871    4524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:06:08.313975 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307875    4524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:06:08.313975 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307877    4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:06:08.313975 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307880    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:06:08.313975 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307882    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:06:08.313975 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307885    4524 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:06:08.313975 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307888    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:06:08.313975 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307891    4524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:06:08.313975 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307899    4524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:06:08.313975 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307902    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:06:08.313975 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307905    4524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307908    4524 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307910    4524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307913    4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307915    4524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.307923    4524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308014    4524 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308021    4524 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308027    4524 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308032    4524 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308036    4524 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308039    4524 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308044    4524 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308048    4524 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308051    4524 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308054    4524 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308058    4524 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308061    4524 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308064    4524 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308067    4524 flags.go:64] FLAG: --cgroup-root=""
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308070    4524 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308074    4524 flags.go:64] FLAG: --client-ca-file=""
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308076    4524 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 19:06:08.314505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308081    4524 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308084    4524 flags.go:64] FLAG: --cluster-domain=""
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308087    4524 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308090    4524 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308094    4524 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308097    4524 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308101    4524 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308105    4524 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308109    4524 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308112    4524 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308115    4524 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308118    4524 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308121    4524 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308124    4524 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308127    4524 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308131    4524 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308135    4524 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308137    4524 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308141    4524 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308143    4524 flags.go:64] FLAG: --enable-server="true"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308146    4524 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308151    4524 flags.go:64] FLAG: --event-burst="100"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308154    4524 flags.go:64] FLAG: --event-qps="50"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308157    4524 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 19:06:08.315298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308160    4524 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308163    4524 flags.go:64] FLAG: --eviction-hard=""
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308180    4524 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308184    4524 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308187    4524 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308190    4524 flags.go:64] FLAG: --eviction-soft=""
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308193    4524 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308196    4524 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308200    4524 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308203    4524 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308206    4524 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308209    4524 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308212    4524 flags.go:64] FLAG: --feature-gates=""
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308216    4524 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308219    4524 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308222    4524 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308226    4524 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308233    4524 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308236    4524 flags.go:64] FLAG: --help="false"
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308239    4524 flags.go:64] FLAG: --hostname-override="ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308242    4524 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308245    4524 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308249    4524 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 19:06:08.315980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308252    4524 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308255    4524 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308258    4524 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308261    4524 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308264    4524 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308267    4524 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308269    4524 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308273    4524 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308276    4524 flags.go:64] FLAG: --kube-reserved=""
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308279    4524 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308282    4524 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308285    4524 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308289    4524 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308292    4524 flags.go:64] FLAG: --lock-file=""
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308295    4524 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308297    4524 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308300    4524 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308311    4524 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308315    4524 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308318    4524 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308321    4524 flags.go:64] FLAG: --logging-format="text"
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308324    4524 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308327    4524 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308330    4524 flags.go:64] FLAG: --manifest-url=""
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308334    4524 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 19:06:08.316608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308338    4524 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308341    4524 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308346    4524 flags.go:64] FLAG: --max-pods="110"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308349    4524 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308352    4524 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308355    4524 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308358    4524 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308361    4524 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308364    4524 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308367    4524 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308374    4524 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308377    4524 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308380    4524 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308384    4524 flags.go:64] FLAG: --pod-cidr=""
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308387    4524 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308390    4524 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308393    4524 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308396    4524 flags.go:64] FLAG: --port="10250"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308399    4524 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308402    4524 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-0d92dc2b650bcfcd5"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308405    4524 flags.go:64] FLAG: --qos-reserved=""
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308408    4524 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308411    4524 flags.go:64] FLAG: --register-node="true"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308414    4524 flags.go:64] FLAG: --register-with-taints=""
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308419    4524 flags.go:64] FLAG: --registry-burst="10"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308422    4524 flags.go:64] FLAG: --registry-qps="5"
Oct 30 19:06:08.317300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308425    4524 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308428    4524 flags.go:64] FLAG: --reserved-memory=""
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308433    4524 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308436    4524 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308439    4524 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308442    4524 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308445    4524 flags.go:64] FLAG: --runonce="false"
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308448    4524 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308451    4524 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308454    4524 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308457    4524 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308460    4524 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308464    4524 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308467    4524 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308470    4524 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308473    4524 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308476    4524 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308479    4524 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308482    4524 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308485    4524 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308488    4524 flags.go:64] FLAG: --system-cgroups=""
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308491    4524 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308496    4524 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308499    4524 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308501    4524 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 19:06:08.318046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308505    4524 flags.go:64] FLAG: --tls-min-version=""
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308508    4524 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308511    4524 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308514    4524 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308517    4524 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308521    4524 flags.go:64] FLAG: --v="2"
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308525    4524 flags.go:64] FLAG: --version="false"
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308529    4524 flags.go:64] FLAG: --vmodule=""
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308533    4524 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.308537    4524 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308667    4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308671    4524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308674    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308676    4524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308679    4524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308682    4524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308685    4524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308688    4524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308690    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308693    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308695    4524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:06:08.318923 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308698    4524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:06:08.319547 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308700    4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:06:08.319547 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308703    4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:06:08.319547 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308706    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:06:08.319547 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308708    4524 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:06:08.319547 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308711    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:06:08.319547 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308714    4524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:06:08.319547 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308716    4524 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:06:08.319547 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308719    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:06:08.319547 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308721    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:06:08.319547 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308724    4524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:06:08.319547 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308726    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:06:08.319547 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308729    4524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:06:08.319547 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308731    4524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:06:08.319547 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308734    4524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:06:08.319547 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308737    4524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:06:08.319547 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308741    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:06:08.319547 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308744    4524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:06:08.319547 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308746    4524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:06:08.319547 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308749    4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:06:08.319547 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308751    4524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:06:08.320104 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308754    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:06:08.320104 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308757    4524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:06:08.320104 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308759    4524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:06:08.320104 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308762    4524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:06:08.320104 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308764    4524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:06:08.320104 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308767    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:06:08.320104 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308769    4524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:06:08.320104 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308772    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:06:08.320104 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308775    4524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:06:08.320104 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308778    4524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:06:08.320104 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308784    4524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:06:08.320104 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308788    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:06:08.320104 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308791    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:06:08.320104 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308793    4524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:06:08.320104 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308796    4524 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:06:08.320104 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308799    4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:06:08.320104 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308802    4524 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:06:08.320104 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308804    4524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:06:08.320104 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308807    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:06:08.320600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308809    4524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:06:08.320600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308812    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:06:08.320600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308815    4524 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:06:08.320600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308817    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:06:08.320600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308820    4524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:06:08.320600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308823    4524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:06:08.320600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308827    4524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:06:08.320600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308831    4524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:06:08.320600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308836    4524 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:06:08.320600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308840    4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:06:08.320600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308844    4524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:06:08.320600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308848    4524 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:06:08.320600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308851    4524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:06:08.320600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308854    4524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:06:08.320600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308856    4524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:06:08.320600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308859    4524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:06:08.320600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308862    4524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:06:08.320600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308865    4524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:06:08.320600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308867    4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:06:08.320600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308870    4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:06:08.321299 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308872    4524 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:06:08.321299 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308875    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:06:08.321299 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308879    4524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:06:08.321299 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308884    4524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:06:08.321299 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308886    4524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:06:08.321299 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308889    4524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:06:08.321299 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308892    4524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:06:08.321299 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308895    4524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:06:08.321299 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308898    4524 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:06:08.321299 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308900    4524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:06:08.321299 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308903    4524 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:06:08.321299 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308906    4524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:06:08.321299 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308908    4524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:06:08.321299 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308911    4524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:06:08.321299 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308914    4524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:06:08.321299 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308916    4524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:06:08.321299 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308919    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:06:08.321299 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308921    4524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:06:08.321299 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308924    4524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:06:08.321299 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308927    4524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:06:08.321902 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308931    4524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:06:08.321902 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308934    4524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:06:08.321902 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308937    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:06:08.321902 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308939    4524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:06:08.321902 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308942    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:06:08.321902 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308944    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:06:08.321902 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308947    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:06:08.321902 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308949    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:06:08.321902 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308952    4524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:06:08.321902 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308955    4524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:06:08.321902 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308958    4524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:06:08.321902 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308963    4524 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:06:08.321902 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308966    4524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:06:08.321902 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308969    4524 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:06:08.321902 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.308972    4524 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:06:08.321902 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.309000    4524 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:06:08.322373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309036    4524 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:06:08.322373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309040    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:06:08.322373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309043    4524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:06:08.322373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309046    4524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:06:08.322373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309048    4524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:06:08.322373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309051    4524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:06:08.322373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309054    4524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:06:08.322373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309057    4524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:06:08.322373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309059    4524 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:06:08.322373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309062    4524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:06:08.322373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309065    4524 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:06:08.322373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309067    4524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:06:08.322373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309070    4524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:06:08.322373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309073    4524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:06:08.322373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309075    4524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:06:08.322373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309078    4524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:06:08.322373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309080    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:06:08.322373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309084    4524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:06:08.322373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309087    4524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:06:08.322373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309089    4524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:06:08.322868 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309092    4524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:06:08.322868 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309094    4524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:06:08.322868 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309097    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:06:08.322868 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309099    4524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:06:08.322868 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309102    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:06:08.322868 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309105    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:06:08.322868 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309107    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:06:08.322868 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309110    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:06:08.322868 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309113    4524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:06:08.322868 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309115    4524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:06:08.322868 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309119    4524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:06:08.322868 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309124    4524 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:06:08.322868 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309128    4524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:06:08.322868 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309131    4524 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:06:08.322868 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309134    4524 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:06:08.322868 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309136    4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:06:08.322868 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309139    4524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:06:08.322868 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309141    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:06:08.322868 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309144    4524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:06:08.322868 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309146    4524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:06:08.323390 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309153    4524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:06:08.323390 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309156    4524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:06:08.323390 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309158    4524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:06:08.323390 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309161    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:06:08.323390 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309163    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:06:08.323390 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309179    4524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:06:08.323390 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309182    4524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:06:08.323390 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309184    4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:06:08.323390 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309187    4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:06:08.323390 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309190    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:06:08.323390 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309193    4524 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:06:08.323390 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309196    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:06:08.323390 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309198    4524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:06:08.323390 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309201    4524 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:06:08.323390 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309204    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:06:08.323390 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309207    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:06:08.323390 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309209    4524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:06:08.323390 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309212    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:06:08.323390 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309214    4524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:06:08.323853 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309217    4524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:06:08.323853 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309219    4524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:06:08.323853 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309222    4524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:06:08.323853 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309224    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:06:08.323853 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309229    4524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:06:08.323853 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309232    4524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:06:08.323853 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309235    4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:06:08.323853 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309238    4524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:06:08.323853 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309240    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:06:08.323853 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309243    4524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:06:08.323853 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309245    4524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:06:08.323853 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309248    4524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:06:08.323853 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309251    4524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:06:08.323853 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309253    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:06:08.323853 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309256    4524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:06:08.323853 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309258    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:06:08.323853 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309261    4524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:06:08.323853 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309263    4524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:06:08.323853 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309267    4524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:06:08.324369 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309271    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:06:08.324369 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309275    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:06:08.324369 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309278    4524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:06:08.324369 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309282    4524 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:06:08.324369 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309285    4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:06:08.324369 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309287    4524 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:06:08.324369 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309290    4524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:06:08.324369 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309293    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:06:08.324369 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309295    4524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:06:08.324369 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309298    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:06:08.324369 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309301    4524 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:06:08.324369 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309304    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:06:08.324369 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309306    4524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:06:08.324369 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309309    4524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:06:08.324369 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309311    4524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:06:08.324369 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309314    4524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:06:08.324369 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309316    4524 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:06:08.324369 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309320    4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:06:08.324369 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309323    4524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:06:08.324369 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309326    4524 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:06:08.324854 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309329    4524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:06:08.324854 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309331    4524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:06:08.324854 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309334    4524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:06:08.324854 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309336    4524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:06:08.324854 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309339    4524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:06:08.324854 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309341    4524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:06:08.324854 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309344    4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:06:08.324854 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.309346    4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:06:08.324854 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.313838    4524 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 19:06:08.324854 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.313854    4524 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 19:06:08.324854 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.313874    4524 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 19:06:08.324854 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.313880    4524 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 19:06:08.324854 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313935    4524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:06:08.324854 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313941    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:06:08.324854 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313945    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:06:08.324854 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313948    4524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:06:08.324854 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313951    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:06:08.324854 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313954    4524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:06:08.324854 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313957    4524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:06:08.325449 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313959    4524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:06:08.325449 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313962    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:06:08.325449 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313965    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:06:08.325449 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313968    4524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:06:08.325449 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313971    4524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:06:08.325449 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313974    4524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:06:08.325449 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313976    4524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:06:08.325449 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313979    4524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:06:08.325449 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313982    4524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:06:08.325449 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313985    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:06:08.325449 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313988    4524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:06:08.325449 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313991    4524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:06:08.325449 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313994    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:06:08.325449 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.313997    4524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:06:08.325449 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314000    4524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:06:08.325449 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314003    4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:06:08.325449 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314005    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:06:08.325449 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314008    4524 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:06:08.325449 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314012    4524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:06:08.325904 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314014    4524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:06:08.325904 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314018    4524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:06:08.325904 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314021    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:06:08.325904 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314025    4524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:06:08.325904 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314028    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:06:08.325904 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314032    4524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:06:08.325904 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314034    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:06:08.325904 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314037    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:06:08.325904 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314039    4524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:06:08.325904 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314042    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:06:08.325904 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314044    4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:06:08.325904 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314047    4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:06:08.325904 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314050    4524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:06:08.325904 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314052    4524 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:06:08.325904 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314055    4524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:06:08.325904 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314058    4524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:06:08.325904 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314060    4524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:06:08.325904 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314063    4524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:06:08.325904 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314066    4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:06:08.326389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314068    4524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:06:08.326389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314072    4524 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:06:08.326389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314074    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:06:08.326389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314077    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:06:08.326389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314079    4524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:06:08.326389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314082    4524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:06:08.326389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314084    4524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:06:08.326389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314088    4524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:06:08.326389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314090    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:06:08.326389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314093    4524 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:06:08.326389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314095    4524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:06:08.326389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314098    4524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:06:08.326389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314101    4524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:06:08.326389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314103    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:06:08.326389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314105    4524 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:06:08.326389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314108    4524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:06:08.326389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314111    4524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:06:08.326389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314113    4524 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:06:08.326389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314116    4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:06:08.326389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314118    4524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:06:08.326875 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314121    4524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:06:08.326875 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314123    4524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:06:08.326875 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314126    4524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:06:08.326875 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314128    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:06:08.326875 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314131    4524 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:06:08.326875 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314133    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:06:08.326875 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314136    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:06:08.326875 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314138    4524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:06:08.326875 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314141    4524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:06:08.326875 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314144    4524 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:06:08.326875 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314146    4524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:06:08.326875 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314149    4524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:06:08.326875 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314151    4524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:06:08.326875 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314154    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:06:08.326875 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314158    4524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:06:08.326875 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314160    4524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:06:08.326875 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314163    4524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:06:08.326875 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314166    4524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:06:08.326875 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314184    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:06:08.327360 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314187    4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:06:08.327360 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314190    4524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:06:08.327360 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314192    4524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:06:08.327360 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314195    4524 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:06:08.327360 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314197    4524 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:06:08.327360 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314200    4524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:06:08.327360 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314202    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:06:08.327360 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314205    4524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:06:08.327360 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314207    4524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:06:08.327360 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314210    4524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:06:08.327360 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314212    4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:06:08.327360 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314214    4524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:06:08.327360 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314217    4524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:06:08.327360 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314219    4524 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:06:08.327360 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314222    4524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:06:08.327360 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314225    4524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:06:08.327360 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314228    4524 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:06:08.327360 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314230    4524 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:06:08.327360 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314233    4524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:06:08.327360 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314235    4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:06:08.327993 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314238    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:06:08.327993 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314240    4524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:06:08.327993 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.314267    4524 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:06:08.327993 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314418    4524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:06:08.327993 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314426    4524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:06:08.327993 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314431    4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:06:08.327993 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314436    4524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:06:08.327993 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314440    4524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:06:08.327993 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314445    4524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:06:08.327993 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314450    4524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:06:08.327993 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314453    4524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:06:08.327993 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314456    4524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:06:08.327993 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314459    4524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:06:08.327993 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314462    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:06:08.327993 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314464    4524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:06:08.328614 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314467    4524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:06:08.328614 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314469    4524 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:06:08.328614 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314472    4524 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:06:08.328614 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314476    4524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:06:08.328614 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314480    4524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:06:08.328614 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314484    4524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:06:08.328614 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314488    4524 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:06:08.328614 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314492    4524 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:06:08.328614 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314496    4524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:06:08.328614 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314500    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:06:08.328614 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314504    4524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:06:08.328614 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314509    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:06:08.328614 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314513    4524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:06:08.328614 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314518    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:06:08.328614 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314522    4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:06:08.328614 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314525    4524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:06:08.328614 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314528    4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:06:08.328614 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314531    4524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:06:08.328614 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314534    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:06:08.328614 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314537    4524 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:06:08.329545 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314540    4524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:06:08.329545 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314543    4524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:06:08.329545 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314546    4524 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:06:08.329545 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314550    4524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:06:08.329545 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314554    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:06:08.329545 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314559    4524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:06:08.329545 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314570    4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:06:08.329545 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314573    4524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:06:08.329545 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314575    4524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:06:08.329545 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314578    4524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:06:08.329545 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314580    4524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:06:08.329545 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314583    4524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:06:08.329545 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314586    4524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:06:08.329545 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314588    4524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:06:08.329545 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314590    4524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:06:08.329545 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314593    4524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:06:08.329545 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314596    4524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:06:08.329545 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314598    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:06:08.329545 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314601    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:06:08.330389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314603    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:06:08.330389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314606    4524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:06:08.330389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314609    4524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:06:08.330389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314611    4524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:06:08.330389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314614    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:06:08.330389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314617    4524 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:06:08.330389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314620    4524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:06:08.330389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314622    4524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:06:08.330389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314625    4524 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:06:08.330389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314627    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:06:08.330389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314630    4524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:06:08.330389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314632    4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:06:08.330389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314635    4524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:06:08.330389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314637    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:06:08.330389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314640    4524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:06:08.330389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314642    4524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:06:08.330389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314645    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:06:08.330389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314647    4524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:06:08.330389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314650    4524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:06:08.330389 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314652    4524 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:06:08.331214 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314655    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:06:08.331214 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314658    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:06:08.331214 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314660    4524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:06:08.331214 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314663    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:06:08.331214 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314665    4524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:06:08.331214 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314668    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:06:08.331214 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314670    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:06:08.331214 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314673    4524 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:06:08.331214 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314675    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:06:08.331214 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314678    4524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:06:08.331214 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314680    4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:06:08.331214 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314683    4524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:06:08.331214 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314685    4524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:06:08.331214 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314687    4524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:06:08.331214 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314690    4524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:06:08.331214 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314693    4524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:06:08.331214 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314695    4524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:06:08.331214 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314698    4524 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:06:08.331214 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314700    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:06:08.331214 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314703    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:06:08.331739 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314705    4524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:06:08.331739 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314708    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:06:08.331739 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314710    4524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:06:08.331739 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314713    4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:06:08.331739 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314715    4524 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:06:08.331739 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314718    4524 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:06:08.331739 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314721    4524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:06:08.331739 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314723    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:06:08.331739 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314726    4524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:06:08.331739 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314728    4524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:06:08.331739 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314731    4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:06:08.331739 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314733    4524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:06:08.331739 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314736    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:06:08.331739 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314739    4524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:06:08.331739 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314741    4524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:06:08.331739 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.314767    4524 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 19:06:08.332212 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314801    4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 19:06:08.332212 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314808    4524 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 19:06:08.332212 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314812    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 19:06:08.332212 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314816    4524 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 19:06:08.332212 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314820    4524 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 19:06:08.332212 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314826    4524 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 19:06:08.332212 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314831    4524 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 19:06:08.332212 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314835    4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 19:06:08.332212 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314839    4524 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 19:06:08.332212 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314843    4524 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 19:06:08.332212 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314849    4524 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 19:06:08.332212 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314854    4524 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 19:06:08.332212 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314860    4524 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 19:06:08.332212 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314863    4524 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 19:06:08.332212 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314867    4524 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 19:06:08.332212 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314872    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 19:06:08.332212 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314875    4524 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 19:06:08.332212 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314879    4524 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 19:06:08.332212 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314883    4524 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 19:06:08.332690 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314887    4524 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 19:06:08.332690 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314891    4524 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 19:06:08.332690 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314894    4524 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 19:06:08.332690 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314901    4524 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 19:06:08.332690 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314905    4524 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 19:06:08.332690 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314909    4524 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 19:06:08.332690 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314913    4524 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 19:06:08.332690 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314917    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 19:06:08.332690 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314921    4524 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 19:06:08.332690 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314925    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 19:06:08.332690 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314931    4524 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 19:06:08.332690 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314936    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 19:06:08.332690 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314940    4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 19:06:08.332690 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314944    4524 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 19:06:08.332690 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314947    4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 19:06:08.332690 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314952    4524 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 19:06:08.332690 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314956    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 19:06:08.332690 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314961    4524 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 19:06:08.332690 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314966    4524 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 19:06:08.332690 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314970    4524 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 19:06:08.333286 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314973    4524 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 19:06:08.333286 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314976    4524 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 19:06:08.333286 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314978    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 19:06:08.333286 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314981    4524 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 19:06:08.333286 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314984    4524 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 19:06:08.333286 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314986    4524 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 19:06:08.333286 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314989    4524 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 19:06:08.333286 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314992    4524 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 19:06:08.333286 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314994    4524 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 19:06:08.333286 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.314997    4524 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 19:06:08.333286 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315000    4524 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 19:06:08.333286 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315002    4524 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 19:06:08.333286 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315005    4524 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 19:06:08.333286 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315008    4524 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 19:06:08.333286 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315010    4524 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 19:06:08.333286 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315014    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 19:06:08.333286 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315017    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 19:06:08.333286 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315020    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 19:06:08.333286 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315022    4524 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 19:06:08.333758 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315025    4524 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 19:06:08.333758 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315027    4524 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 19:06:08.333758 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315030    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 19:06:08.333758 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315034    4524 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 19:06:08.333758 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315037    4524 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 19:06:08.333758 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315040    4524 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 19:06:08.333758 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315043    4524 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 19:06:08.333758 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315047    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 19:06:08.333758 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315050    4524 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 19:06:08.333758 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315053    4524 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 19:06:08.333758 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315055    4524 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 19:06:08.333758 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315058    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 19:06:08.333758 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315061    4524 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 19:06:08.333758 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315063    4524 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 19:06:08.333758 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315066    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 19:06:08.333758 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315068    4524 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 19:06:08.333758 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315071    4524 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 19:06:08.333758 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315073    4524 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 19:06:08.333758 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315076    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 19:06:08.333758 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315078    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 19:06:08.334304 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315081    4524 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 19:06:08.334304 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315083    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 19:06:08.334304 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315086    4524 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 19:06:08.334304 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315089    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 19:06:08.334304 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315091    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 19:06:08.334304 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315094    4524 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 19:06:08.334304 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315096    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 19:06:08.334304 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315099    4524 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 19:06:08.334304 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315103    4524 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 19:06:08.334304 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315106    4524 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 19:06:08.334304 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315108    4524 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 19:06:08.334304 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315111    4524 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 19:06:08.334304 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315113    4524 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 19:06:08.334304 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315116    4524 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 19:06:08.334304 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315119    4524 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 19:06:08.334304 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315123    4524 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 19:06:08.334304 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315126    4524 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 19:06:08.334304 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315128    4524 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 19:06:08.334304 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315132    4524 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 19:06:08.334304 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315135    4524 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 19:06:08.334779 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315138    4524 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 19:06:08.334779 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315140    4524 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 19:06:08.334779 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315143    4524 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 19:06:08.334779 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315145    4524 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 19:06:08.334779 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315148    4524 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 19:06:08.334779 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315150    4524 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 19:06:08.334779 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315153    4524 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 19:06:08.334779 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:08.315155    4524 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 19:06:08.334779 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.315231    4524 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 19:06:08.334779 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.316244    4524 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 19:06:08.334779 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.316307    4524 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 19:06:08.334779 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.316480    4524 server.go:1014] "Starting client certificate rotation"
Oct 30 19:06:08.334779 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.316627    4524 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 19:06:08.334779 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.316681    4524 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 19:00:27 +0000 UTC" deadline="2028-04-28 07:46:26.697488575 +0000 UTC"
Oct 30 19:06:08.334779 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.316714    4524 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13092h40m18.380776417s"
Oct 30 19:06:08.334779 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.316910    4524 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 19:06:08.335192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.317921    4524 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 19:06:08.335192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.319434    4524 log.go:25] "Validated CRI v1 runtime API"
Oct 30 19:06:08.335192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.320963    4524 log.go:25] "Validated CRI v1 image API"
Oct 30 19:06:08.335192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.321620    4524 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 19:06:08.335192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.323091    4524 fs.go:135] Filesystem UUIDs: map[4be381ea-d5f6-46b6-b428-26d1475b7e1b:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 c6e9cc3a-88a0-4416-9e5f-7597206b67a3:/dev/nvme0n1p4]
Oct 30 19:06:08.335337 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.323116    4524 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/010e578cb4471120dd639ca96c7e1d98938702738350708dc15596dac065894c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/010e578cb4471120dd639ca96c7e1d98938702738350708dc15596dac065894c/userdata/shm major:0 minor:92 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1b6e265adf5cb80c3712e09bc95d489b247261151cb0892a6e6f2e85afb2e715/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1b6e265adf5cb80c3712e09bc95d489b247261151cb0892a6e6f2e85afb2e715/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1c1ff350bbc22b21f236f6650a0e079cf70a79cac5f1f0878f7ac1a518b4d21e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1c1ff350bbc22b21f236f6650a0e079cf70a79cac5f1f0878f7ac1a518b4d21e/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2847ba93a9d1fec6ee5a390815f406cee1525845709e5b52dea049ee3a0ab759/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2847ba93a9d1fec6ee5a390815f406cee1525845709e5b52dea049ee3a0ab759/userdata/shm major:0 minor:226 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3421b7bfa5bcba71023eedf52a8df24102943c22620eb2e3400eaf57f5846207/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3421b7bfa5bcba71023eedf52a8df24102943c22620eb2e3400eaf57f5846207/userdata/shm major:0 minor:219 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3fe2c2a4e97d682721b15997a4fa394a0bcb7da534ba776f54e789550e53d96b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3fe2c2a4e97d682721b15997a4fa394a0bcb7da534ba776f54e789550e53d96b/userdata/shm major:0 minor:213 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5f0fc158d89f6d16c311fafc38d332790bdbfda0144023ef8074b2a4aeaa76a5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5f0fc158d89f6d16c311fafc38d332790bdbfda0144023ef8074b2a4aeaa76a5/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8c20c3f66ae4d81e1612ec8a561e5e595426adad4dead32438fcc369135a765f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8c20c3f66ae4d81e1612ec8a561e5e595426adad4dead32438fcc369135a765f/userdata/shm major:0 minor:84 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a2167fb8c16acfe29480b3b313e7b0c993c3a07c9aa231868e3cfc7c3e909333/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a2167fb8c16acfe29480b3b313e7b0c993c3a07c9aa231868e3cfc7c3e909333/userdata/shm major:0 minor:73 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a3f490ce9eb80385ff59eaf9e69bee4549d03457e691558955ca4e45456ec0d7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a3f490ce9eb80385ff59eaf9e69bee4549d03457e691558955ca4e45456ec0d7/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a695b8224522047769b03fbca685bcd7ba26b1b00fc365e370099af9cc14b4d3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a695b8224522047769b03fbca685bcd7ba26b1b00fc365e370099af9cc14b4d3/userdata/shm major:0 minor:225 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b83b2f591bd7deef664e4e37dbd9804306d3afe4efeb9f587032c95e3e194dfd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b83b2f591bd7deef664e4e37dbd9804306d3afe4efeb9f587032c95e3e194dfd/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bf9224e80094e9d2e45da0443ed7c91b2e99fe2da568a6619c8c63306517c3f7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bf9224e80094e9d2e45da0443ed7c91b2e99fe2da568a6619c8c63306517c3f7/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c5816817ad4d94d47e364d6a051169d0f69b695fc2f17e86ec0af6250b70843b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c5816817ad4d94d47e364d6a051169d0f69b695fc2f17e86ec0af6250b70843b/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d1edc3f3a2dcdf669197742726ce951eb7f03990d787797f1a4a4c32b9085448/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d1edc3f3a2dcdf669197742726ce951eb7f03990d787797f1a4a4c32b9085448/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e00ec3b69fb49a9276c87d5dc5a8dd5226a2db07f48ab9f9d4781dcf0c682ccc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e00ec3b69fb49a9276c87d5dc5a8dd5226a2db07f48ab9f9d4781dcf0c682ccc/userdata/shm major:0 minor:208 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/260d4781-1e14-4a9a-8840-49c1f9bdbfbe/volumes/kubernetes.io~projected/kube-api-access-9nlrh:{mountpoint:/var/lib/kubelet/pods/260d4781-1e14-4a9a-8840-49c1f9bdbfbe/volumes/kubernetes.io~projected/kube-api-access-9nlrh major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2ac4e9e2-d815-4b3b-86cd-db537f984e6c/volumes/kubernetes.io~projected/kube-api-access-wqgn5:{mountpoint:/var/lib/kubelet/pods/2ac4e9e2-d815-4b3b-86cd-db537f984e6c/volumes/kubernetes.io~projected/kube-api-access-wqgn5 major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30fc2aa0-31d5-4577-8064-25dbb6cea5d7/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/30fc2aa0-31d5-4577-8064-25dbb6cea5d7/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30fc2aa0-31d5-4577-8064-25dbb6cea5d7/volumes/kubernetes.io~projected/kube-api-access-qxvt7:{mountpoint:/var/lib/kubelet/pods/30fc2aa0-31d5-4577-8064-25dbb6cea5d7/volumes/kubernetes.io~projected/kube-api-access-qxvt7 major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30fc2aa0-31d5-4577-8064-25dbb6cea5d7/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/30fc2aa0-31d5-4577-8064-25dbb6cea5d7/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5fdb2ea2-a56c-46f0-b155-468303552965/volumes/kubernetes.io~projected/kube-api-access-qw4hs:{mountpoint:/var/lib/kubelet/pods/5fdb2ea2-a56c-46f0-b155-468303552965/volumes/kubernetes.io~projected/kube-api-access-qw4hs major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7411c16b-533b-4152-bbe6-52262afa37da/volumes/kubernetes.io~projected/kube-api-access-wc2g2:{mountpoint:/var/lib/kubelet/pods/7411c16b-533b-4152-bbe6-52262afa37da/volumes/kubernetes.io~projected/kube-api-access-wc2g2 major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7411c16b-533b-4152-bbe6-52262afa37da/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/7411c16b-533b-4152-bbe6-52262afa37da/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/838f8f5e-ac73-4280-8ed3-173a51c42b9e/volumes/kubernetes.io~projected/kube-api-access-zpgcn:{mountpoint:/var/lib/kubelet/pods/838f8f5e-ac73-4280-8ed3-173a51c42b9e/volumes/kubernetes.io~projected/kube-api-access-zpgcn major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8d23c4bb-0012-4e09-bc59-c6689f9ea394/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/8d23c4bb-0012-4e09-bc59-c6689f9ea394/volumes/kubernetes.io~secret/agent-certs major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/94709da2-704a-421c-aa2f-1844db5bec96/volumes/kubernetes.io~projected/kube-api-access-6hf6x:{mountpoint:/var/lib/kubelet/pods/94709da2-704a-421c-aa2f-1844db5bec96/volumes/kubernetes.io~projected/kube-api-access-6hf6x major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b7c1242a-b12d-49b2-8052-781394f01418/volumes/kubernetes.io~projected/kube-api-access-qvx2d:{mountpoint:/var/lib/kubelet/pods/b7c1242a-b12d-49b2-8052-781394f01418/volumes/kubernetes.io~projected/kube-api-access-qvx2d major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ba82c577-b89d-44f7-87e4-2d00d1e2a5de/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/ba82c577-b89d-44f7-87e4-2d00d1e2a5de/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c1cc12f1-be52-4a9f-a8a2-53fe3c493431/volumes/kubernetes.io~projected/kube-api-access-8pq2q:{mountpoint:/var/lib/kubelet/pods/c1cc12f1-be52-4a9f-a8a2-53fe3c493431/volumes/kubernetes.io~projected/kube-api-access-8pq2q major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c4b9f9bd-6097-4524-9c86-44c7f484f278/volumes/kubernetes.io~projected/kube-api-access-6jmzz:{mountpoint:/var/lib/kubelet/pods/c4b9f9bd-6097-4524-9c86-44c7f484f278/volumes/kubernetes.io~projected/kube-api-access-6jmzz major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a/volumes/kubernetes.io~projected/kube-api-access-65lrg:{mountpoint:/var/lib/kubelet/pods/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a/volumes/kubernetes.io~projected/kube-api-access-65lrg major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce07e9b3-5e98-4d16-8158-afba6fe42281/volumes/kubernetes.io~projected/kube-api-access-wxhpm:{mountpoint:/var/lib/kubelet/pods/ce07e9b3-5e98-4d16-8158-afba6fe42281/volumes/kubernetes.io~projected/kube-api-access-wxhpm major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d36c6f3a-c76b-4010-898b-9cd567933c5a/volumes/kubernetes.io~projected/kube-api-access-vxkwv:{mountpoint:/var/lib/kubelet/pods/d36c6f3a-c76b-4010-898b-9cd567933c5a/volumes/kubernetes.io~projected/kube-api-access-vxkwv major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d36c6f3a-c76b-4010-898b-9cd567933c5a/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/d36c6f3a-c76b-4010-898b-9cd567933c5a/volumes/kubernetes.io~secret/signing-key major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ed4e7b61-d816-49fc-b4d9-36236a716227/volumes/kubernetes.io~projected/kube-api-access-b4hr8:{mountpoint:/var/lib/kubelet/pods/ed4e7b61-d816-49fc-b4d9-36236a716227/volumes/kubernetes.io~projected/kube-api-access-b4hr8 major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fd24c416-9f3c-4955-a536-08f20ccdbbbb/volumes/kubernetes.io~projected/kube-api-access-49bwq:{mountpoint:/var/lib/kubelet/pods/fd24c416-9f3c-4955-a536-08f20ccdbbbb/volumes/kubernetes.io~projected/kube-api-access-49bwq major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1/volumes/kubernetes.io~empty-dir/tmp major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1/volumes/kubernetes.io~projected/kube-api-access-qdsdn:{mountpoint:/var/lib/kubelet/pods/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1/volumes/kubernetes.io~projected/kube-api-access-qdsdn major:0 minor:64 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/fc79d3881adecfb6ca2238ebd725bbb564328b8ba8ba4bde85a3c8f9e1538b22/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/177959894da5c581742c7f025e418e27f825a6cd78c6bd06e07e1d458017e28c/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/2af220a952e2372ec1d672e46460cdd2e895b5396bfe7b7e54106d1134f95a5c/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/52e5fa367e728168c511136c4f715b6cb1310f901186e7b1b645ba2a26eaa235/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/e5e83df202e04574dabf50b0c63350f244c77d0cab5d61daae3060bd68b3a9cc/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/928e68e6b64110f15dccacfd2f36a985a0202ae7ecb94d1ad284f05bc4f2a77c/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/7ced9d4212bf87bc306c8a6be0265d2495bd736b6533c71e8530d2d61efa0f36/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/1f6921240c13bf5e9cf3ea26b874a41ddbbdcccba2594467def0df5e61ce3a67/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/91edd4e9917ecafe1ae7aa7f24832c7ddad042021247f11fd24fd05a87fbae3b/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/86739bc278dbc802fa515738cfb3a9fc76290cbca941804fe8a54f489a715dc9/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/98926918c88805a448e3564893143a7ae37d9ffa36060c6e8b45306ada4af43b/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/40b7edfc8252086c1485a0def8f324f9ed441e07fbf7dd6dbfeff7dc2462a620/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-149:{mountpoint:/var/lib/containers/storage/overlay/83cc2ff7beefa4b3c298037f6093bfce53d169084e38480df4511f3fd2ca7fe6/merged major:0 minor:149 fsType:overlay blockSize:0} overlay_0-154:{mountpoint:/var/lib/containers/storage/overlay/86c130374cf1e528047b408708dd2ef4111051ae050052b3994d0043d7c32c4e/merged major:0 minor:154 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/db934e5f839e69747f0e3efec52374f8a07b53b7696b4f3848372e4bc60302ed/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/741e0e38e51ec2872dc82f1d0721b6d0088cca74596554824ae4aaea3ff068a9/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/f17814eb080344e47d4174eb875f7a455f7c7cb7b994c8b6f0628ade68c0cef2/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/a2dde27322f4c46cb2e4bf2d9e5bb6514a44106886c2c2e0888dbf400d08ad21/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/c3e19e98acc4d4bffd82e56e42be7932871a363a763ae6142f8da29d5612892f/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/605f7dda8cde525bc5a30b307de59ae263899715c20542f3ab0859b2897406ab/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/e8cacf99fae2e8d87a4d853826c784641c5cf087093c4d9494f2b19984b89e79/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-204:{mountpoint:/var/lib/containers/storage/overlay/49a8ea87026f7d1c34b31df4e5cf17666cb32505b75f837f84ba501ab9845b21/merged major:0 minor:204 fsType:overlay blockSize:0} overlay_0-210:{mountpoint:/var/lib/containers/storage/overlay/c6c46101f2acd379b9276a43c9f991e98be3d82a788752b067b1a770f783305c/merged major:0 minor:210 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/16293c172a6cf3f0a507ca2a805860214f638aeb1f9946ac8965b16ed569464b/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-222:{mountpoint:/var/lib/containers/storage/overlay/bb23f7d4a5c27de7500453e31f9eea9aa5342e3d503c5a239e241758ca4fac2b/merged major:0 minor:222 fsType:overlay blockSize:0} overlay_0-229:{mountpoint:/var/lib/containers/storage/overlay/9e978899dfcf38bc48632fd81b34bd069d46bdf16af1af4ee94d07ea0d648d00/merged major:0 minor:229 fsType:overlay blockSize:0} overlay_0-231:{mountpoint:/var/lib/containers/storage/overlay/eba8107b3dc52b6bba27c36c04ad577bb0f07d693bce7c849dd88fb86ba43741/merged major:0 minor:231 fsType:overlay blockSize:0} overlay_0-237:{mountpoint:/var/lib/containers/storage/overlay/2050a69e5cfe4306136a2f2707b597277e7b3611f4d3d51048821d46a98bbe1e/merged major:0 minor:237 fsType:overlay blockSize:0} overlay_0-239:{mountpoint:/var/lib/containers/storage/overlay/f422d987c882ed4806acf678c5a7e145b0464180d911d3abaefcc5128627e46f/merged major:0 minor:239 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/a556ba1b2293f0317927024fb6e1c933c41824f204bd13537b4fec479a3b2579/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/55ea04244481b90fd4e67ef090e0436f1df57edfe3958055550e2e088697e774/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/5c46c646e12f267ac8c516af294fb029316404fa62da894faee9a16097f3d635/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/4516c367677e265d0e19acfd3ca461a03cc223d3165f8f119e5bc5913c8d72c1/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/9f83a9225b5380ce8ce2df9bc89f74854ab979094850b1627915a88dd2d1df32/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/7f5940344c84c9574307ca4e0ad6bce4039eed8b153d8ec9f3acdbb291ba73e6/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-71:{mountpoint:/var/lib/containers/storage/overlay/f5662bd3cad2f31b85f66a2d5b2e62de7baebb55e574a7414ab31f8ec349d284/merged major:0 minor:71 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/40ea10b3732fef3983878784179b15b7f99456a07d7fbba843f069aa4f0b919c/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-79:{mountpoint:/var/lib/containers/storage/overlay/2dd26f510229ad4fc82a915f11850f6c0f71e7fb18b9d568d96be99fe2061428/merged major:0 minor:79 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/c96584274c31480c21724706096c5525ba7a021c4ccbe62427e44488775cf709/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/291e030825ca3997169df05b5743d132d88a281e2d795bea34e5da7f20c885c0/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/7443d3e35cea712f9641797dc4b7c22034447a1f9051be53884a973f05e63aba/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/d7b5c724152d354b3f486cbb07f12a24e0a0692888152ae891ce5e9493667ef0/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/f9bddc621db9925d0856e6bf94f68954ce84fe7ac1f1e043e97bf0f9d024d2bb/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.330511    4524 manager.go:217] Machine: {Timestamp:2026-10-30 19:06:08.329709558 +0000 UTC m=+0.070700629 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3109759 MemoryCapacity:33164341248 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22775fc95798b59cd9a0c3008b4bc0 SystemUUID:ec22775f-c957-98b5-9cd9-a0c3008b4bc0 BootID:ac0e3e80-ac2b-483f-8b1c-61d08d4afbbd Filesystems:[{Device:overlay_0-239 DeviceMajor:0 DeviceMinor:239 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b7c1242a-b12d-49b2-8052-781394f01418/volumes/kubernetes.io~projected/kube-api-access-qvx2d DeviceMajor:0 DeviceMinor:56 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:61 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-79 DeviceMajor:0 DeviceMinor:79 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5f0fc158d89f6d16c311fafc38d332790bdbfda0144023ef8074b2a4aeaa76a5/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c1cc12f1-be52-4a9f-a8a2-53fe3c493431/volumes/kubernetes.io~projected/kube-api-access-8pq2q DeviceMajor:0 DeviceMinor:224 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fd24c416-9f3c-4955-a536-08f20ccdbbbb/volumes/kubernetes.io~projected/kube-api-access-49bwq DeviceMajor:0 DeviceMinor:65 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a3f490ce9eb80385ff59eaf9e69bee4549d03457e691558955ca4e45456ec0d7/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7411c16b-533b-4152-bbe6-52262afa37da/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/c4b9f9bd-6097-4524-9c86-44c7f484f278/volumes/kubernetes.io~projected/kube-api-access-6jmzz DeviceMajor:0 DeviceMinor:207 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582172672 Type:vfs Inodes:1048576 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8c20c3f66ae4d81e1612ec8a561e5e595426adad4dead32438fcc369135a765f/userdata/shm DeviceMajor:0 DeviceMinor:84 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bf9224e80094e9d2e45da0443ed7c91b2e99fe2da568a6619c8c63306517c3f7/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-222 DeviceMajor:0 DeviceMinor:222 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a695b8224522047769b03fbca685bcd7ba26b1b00fc365e370099af9cc14b4d3/userdata/shm DeviceMajor:0 DeviceMinor:225 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/94709da2-704a-421c-aa2f-1844db5bec96/volumes/kubernetes.io~projected/kube-api-access-6hf6x DeviceMajor:0 DeviceMinor:57 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/838f8f5e-ac73-4280-8ed3-173a51c42b9e/volumes/kubernetes.io~projected/kube-api-access-zpgcn DeviceMajor:0 DeviceMinor:59 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:62 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d1edc3f3a2dcdf669197742726ce951eb7f03990d787797f1a4a4c32b9085448/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1c1ff350bbc22b21f236f6650a0e079cf70a79cac5f1f0878f7ac1a518b4d21e/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2847ba93a9d1fec6ee5a390815f406cee1525845709e5b52dea049ee3a0ab759/userdata/shm DeviceMajor:0 DeviceMinor:226 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/260d4781-1e14-4a9a-8840-49c1f9bdbfbe/volumes/kubernetes.io~projected/kube-api-access-9nlrh DeviceMajor:0 DeviceMinor:67 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2ac4e9e2-d815-4b3b-86cd-db537f984e6c/volumes/kubernetes.io~projected/kube-api-access-wqgn5 DeviceMajor:0 DeviceMinor:206 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3421b7bfa5bcba71023eedf52a8df24102943c22620eb2e3400eaf57f5846207/userdata/shm DeviceMajor:0 DeviceMinor:219 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-229 DeviceMajor:0 DeviceMinor:229 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3fe2c2a4e97d682721b15997a4fa394a0bcb7da534ba776f54e789550e53d96b/userdata/shm DeviceMajor:0 DeviceMinor:213 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/30fc2aa0-31d5-4577-8064-25dbb6cea5d7/volumes/kubernetes.io~projected/kube-api-access-qxvt7 DeviceMajor:0 DeviceMinor:66 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-231 DeviceMajor:0 DeviceMinor:231 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/010e578cb4471120dd639ca96c7e1d98938702738350708dc15596dac065894c/userdata/shm DeviceMajor:0 DeviceMinor:92 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7411c16b-533b-4152-bbe6-52262afa37da/volumes/kubernetes.io~projected/kube-api-access-wc2g2 DeviceMajor:0 DeviceMinor:197 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-237 DeviceMajor:0 DeviceMinor:237 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a/volumes/kubernetes.io~projected/kube-api-access-65lrg DeviceMajor:0 DeviceMinor:55 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8d23c4bb-0012-4e09-bc59-c6689f9ea394/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:60 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a2167fb8c16acfe29480b3b313e7b0c993c3a07c9aa231868e3cfc7c3e909333/userdata/shm DeviceMajor:0 DeviceMinor:73 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-149 DeviceMajor:0 DeviceMinor:149 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c5816817ad4d94d47e364d6a051169d0f69b695fc2f17e86ec0af6250b70843b/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b83b2f591bd7deef664e4e37dbd9804306d3afe4efeb9f587032c95e3e194dfd/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/30fc2aa0-31d5-4577-8064-25dbb6cea5d7/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/d36c6f3a-c76b-4010-898b-9cd567933c5a/volumes/kubernetes.io~projected/kube-api-access-vxkwv DeviceMajor:0 DeviceMinor:218 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-71 DeviceMajor:0 DeviceMinor:71 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-204 DeviceMajor:0 DeviceMinor:204 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5fdb2ea2-a56c-46f0-b155-468303552965/volumes/kubernetes.io~projected/kube-api-access-qw4hs DeviceMajor:0 DeviceMinor:212 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1b6e265adf5cb80c3712e09bc95d489b247261151cb0892a6e6f2e85afb2e715/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-154 DeviceMajor:0 DeviceMinor:154 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ce07e9b3-5e98-4d16-8158-afba6fe42281/volumes/kubernetes.io~projected/kube-api-access-wxhpm DeviceMajor:0 DeviceMinor:201 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1/volumes/kubernetes.io~projected/kube-api-access-qdsdn DeviceMajor:0 DeviceMinor:64 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-210 DeviceMajor:0 DeviceMinor:210 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d36c6f3a-c76b-4010-898b-9cd567933c5a/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:217 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/30fc2aa0-31d5-4577-8064-25dbb6cea5d7/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:63 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e00ec3b69fb49a9276c87d5dc5a8dd5226a2db07f48ab9f9d4781dcf0c682ccc/userdata/shm DeviceMajor:0 DeviceMinor:208 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ed4e7b61-d816-49fc-b4d9-36236a716227/volumes/kubernetes.io~projected/kube-api-access-b4hr8 DeviceMajor:0 DeviceMinor:58 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ba82c577-b89d-44f7-87e4-2d00d1e2a5de/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:221 Capacity:31985741824 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:1c1ff350bbc22b2 MacAddress:02:4d:6a:9c:ab:cd Speed:10000 Mtu:8901} {Name:2847ba93a9d1fec MacAddress:fe:1b:a9:b9:e2:4a Speed:10000 Mtu:8901} {Name:3421b7bfa5bcba7 MacAddress:32:4b:82:f2:3f:4e Speed:10000 Mtu:8901} {Name:3fe2c2a4e97d682 MacAddress:c2:28:3c:6b:d5:65 Speed:10000 Mtu:8901} {Name:a695b8224522047 MacAddress:36:2b:20:d4:61:d0 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:fd:f4:f4:53 Speed:0 Mtu:9001} {Name:br-int MacAddress:0e:39:f6:0f:87:71 Speed:0 Mtu:8901} {Name:e00ec3b69fb49a9 MacAddress:ce:e1:9e:54:c2:42 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:fd:f4:f4:53 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:02:8f:6a:95:c1:8c Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:82:23:10:48:0b:5b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164341248 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.330990    4524 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.331047    4524 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.331323    4524 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.331349    4524 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-200.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.331463    4524 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.331471    4524 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.331482    4524 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.331494    4524 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.331572    4524 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.331714    4524 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.331817    4524 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.331849    4524 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.331858    4524 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.331876    4524 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.331887    4524 kubelet.go:405] "Adding apiserver pod source"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.331895    4524 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.332620    4524 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.332633    4524 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.333101    4524 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.334984    4524 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.334993    4524 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.335091    4524 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.335278    4524 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.335305    4524 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.335414    4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.335427    4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.335433    4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.335438    4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.335444    4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.335453    4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.335462    4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.335470    4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.335477    4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.335483    4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.335496    4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 19:06:08.335489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.335505    4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 19:06:08.336603 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.335531    4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 19:06:08.336603 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.335541    4524 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 19:06:08.338909 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.338895    4524 server.go:1263] "Started kubelet"
Oct 30 19:06:08.342432 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.339121    4524 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 19:06:08.342432 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.339103    4524 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 19:06:08.342432 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.339459    4524 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 19:06:08.342432 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.339776    4524 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 19:06:08.342432 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.340997    4524 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 19:06:08.339496 ip-10-0-137-200 systemd[1]: Started Kubernetes Kubelet.
Oct 30 19:06:08.352077 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.351706    4524 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 19:06:08.354146 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.352363    4524 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 19:06:08.354146 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:06:08.352981    4524 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-137-200.us-east-2.compute.internal\" not found"
Oct 30 19:06:08.354146 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.353030    4524 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 19:06:08.354146 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.353037    4524 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 19:06:08.354146 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.353081    4524 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 19:06:08.354146 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.353794    4524 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 19:00:29 +0000 UTC" deadline="2028-04-06 23:26:44.803181752 +0000 UTC"
Oct 30 19:06:08.354146 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.353834    4524 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12580h20m36.44935123s"
Oct 30 19:06:08.354146 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.354029    4524 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 19:06:08.354146 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.354043    4524 factory.go:55] Registering systemd factory
Oct 30 19:06:08.354146 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.354053    4524 factory.go:223] Registration of the systemd container factory successfully
Oct 30 19:06:08.354640 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.354333    4524 factory.go:153] Registering CRI-O factory
Oct 30 19:06:08.354640 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.354350    4524 factory.go:223] Registration of the crio container factory successfully
Oct 30 19:06:08.354640 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.354369    4524 factory.go:103] Registering Raw factory
Oct 30 19:06:08.354640 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.354405    4524 manager.go:1199] Started watching for new ooms in manager
Oct 30 19:06:08.355124 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.355082    4524 manager.go:319] Starting recovery of all containers
Oct 30 19:06:08.358150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.358102    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="260d4781-1e14-4a9a-8840-49c1f9bdbfbe" volumeName="kubernetes.io/projected/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-kube-api-access-9nlrh" seLinuxMountContext=""
Oct 30 19:06:08.358336 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.358317    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ac4e9e2-d815-4b3b-86cd-db537f984e6c" volumeName="kubernetes.io/configmap/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-service-ca-bundle" seLinuxMountContext=""
Oct 30 19:06:08.358474 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.358459    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ac4e9e2-d815-4b3b-86cd-db537f984e6c" volumeName="kubernetes.io/projected/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-kube-api-access-wqgn5" seLinuxMountContext=""
Oct 30 19:06:08.358606 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.358563    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7411c16b-533b-4152-bbe6-52262afa37da" volumeName="kubernetes.io/secret/7411c16b-533b-4152-bbe6-52262afa37da-cert" seLinuxMountContext=""
Oct 30 19:06:08.358708 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.358694    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94709da2-704a-421c-aa2f-1844db5bec96" volumeName="kubernetes.io/projected/94709da2-704a-421c-aa2f-1844db5bec96-kube-api-access-6hf6x" seLinuxMountContext=""
Oct 30 19:06:08.358827 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.358810    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4b9f9bd-6097-4524-9c86-44c7f484f278" volumeName="kubernetes.io/projected/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-api-access-6jmzz" seLinuxMountContext=""
Oct 30 19:06:08.358918 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.358902    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ac4e9e2-d815-4b3b-86cd-db537f984e6c" volumeName="kubernetes.io/empty-dir/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-archives-path" seLinuxMountContext=""
Oct 30 19:06:08.359024 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.359010    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30fc2aa0-31d5-4577-8064-25dbb6cea5d7" volumeName="kubernetes.io/configmap/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-ovnkube-config" seLinuxMountContext=""
Oct 30 19:06:08.359114 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.359101    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7411c16b-533b-4152-bbe6-52262afa37da" volumeName="kubernetes.io/projected/7411c16b-533b-4152-bbe6-52262afa37da-kube-api-access-wc2g2" seLinuxMountContext=""
Oct 30 19:06:08.359211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.359197    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8d23c4bb-0012-4e09-bc59-c6689f9ea394" volumeName="kubernetes.io/secret/8d23c4bb-0012-4e09-bc59-c6689f9ea394-agent-certs" seLinuxMountContext=""
Oct 30 19:06:08.359300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.359286    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b7c1242a-b12d-49b2-8052-781394f01418" volumeName="kubernetes.io/configmap/b7c1242a-b12d-49b2-8052-781394f01418-cni-binary-copy" seLinuxMountContext=""
Oct 30 19:06:08.359380 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.359368    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ed4e7b61-d816-49fc-b4d9-36236a716227" volumeName="kubernetes.io/projected/ed4e7b61-d816-49fc-b4d9-36236a716227-kube-api-access-b4hr8" seLinuxMountContext=""
Oct 30 19:06:08.359457 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.359442    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8d23c4bb-0012-4e09-bc59-c6689f9ea394" volumeName="kubernetes.io/configmap/8d23c4bb-0012-4e09-bc59-c6689f9ea394-konnectivity-ca" seLinuxMountContext=""
Oct 30 19:06:08.359534 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.359521    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b7c1242a-b12d-49b2-8052-781394f01418" volumeName="kubernetes.io/projected/b7c1242a-b12d-49b2-8052-781394f01418-kube-api-access-qvx2d" seLinuxMountContext=""
Oct 30 19:06:08.359636 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.359619    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a" volumeName="kubernetes.io/projected/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-kube-api-access-65lrg" seLinuxMountContext=""
Oct 30 19:06:08.359733 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.359719    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1" volumeName="kubernetes.io/empty-dir/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-tuned" seLinuxMountContext=""
Oct 30 19:06:08.359822 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.359797    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30fc2aa0-31d5-4577-8064-25dbb6cea5d7" volumeName="kubernetes.io/configmap/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 19:06:08.359905 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.359892    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a" volumeName="kubernetes.io/configmap/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-daemon-config" seLinuxMountContext=""
Oct 30 19:06:08.359995 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.359968    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce07e9b3-5e98-4d16-8158-afba6fe42281" volumeName="kubernetes.io/configmap/ce07e9b3-5e98-4d16-8158-afba6fe42281-config-volume" seLinuxMountContext=""
Oct 30 19:06:08.360121 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.359991    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce07e9b3-5e98-4d16-8158-afba6fe42281" volumeName="kubernetes.io/empty-dir/ce07e9b3-5e98-4d16-8158-afba6fe42281-tmp-dir" seLinuxMountContext=""
Oct 30 19:06:08.360121 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.360030    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce07e9b3-5e98-4d16-8158-afba6fe42281" volumeName="kubernetes.io/projected/ce07e9b3-5e98-4d16-8158-afba6fe42281-kube-api-access-wxhpm" seLinuxMountContext=""
Oct 30 19:06:08.360121 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.360044    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ac4e9e2-d815-4b3b-86cd-db537f984e6c" volumeName="kubernetes.io/configmap/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 19:06:08.360121 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.360057    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="94709da2-704a-421c-aa2f-1844db5bec96" volumeName="kubernetes.io/empty-dir/94709da2-704a-421c-aa2f-1844db5bec96-tmp-dir" seLinuxMountContext=""
Oct 30 19:06:08.360121 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.360073    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b7c1242a-b12d-49b2-8052-781394f01418" volumeName="kubernetes.io/configmap/b7c1242a-b12d-49b2-8052-781394f01418-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 19:06:08.360121 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.360086    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c1cc12f1-be52-4a9f-a8a2-53fe3c493431" volumeName="kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q" seLinuxMountContext=""
Oct 30 19:06:08.360121 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.360099    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a" volumeName="kubernetes.io/configmap/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-cni-binary-copy" seLinuxMountContext=""
Oct 30 19:06:08.360121 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.360113    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1" volumeName="kubernetes.io/projected/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-kube-api-access-qdsdn" seLinuxMountContext=""
Oct 30 19:06:08.360121 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.360125    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="838f8f5e-ac73-4280-8ed3-173a51c42b9e" volumeName="kubernetes.io/configmap/838f8f5e-ac73-4280-8ed3-173a51c42b9e-serviceca" seLinuxMountContext=""
Oct 30 19:06:08.360624 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.360143    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30fc2aa0-31d5-4577-8064-25dbb6cea5d7" volumeName="kubernetes.io/configmap/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-env-overrides" seLinuxMountContext=""
Oct 30 19:06:08.360624 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.360183    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30fc2aa0-31d5-4577-8064-25dbb6cea5d7" volumeName="kubernetes.io/projected/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-kube-api-access-qxvt7" seLinuxMountContext=""
Oct 30 19:06:08.360624 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.360197    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30fc2aa0-31d5-4577-8064-25dbb6cea5d7" volumeName="kubernetes.io/secret/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 19:06:08.360624 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.360210    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b7c1242a-b12d-49b2-8052-781394f01418" volumeName="kubernetes.io/configmap/b7c1242a-b12d-49b2-8052-781394f01418-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 19:06:08.360624 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.360223    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ba82c577-b89d-44f7-87e4-2d00d1e2a5de" volumeName="kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret" seLinuxMountContext=""
Oct 30 19:06:08.360624 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.360248    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4b9f9bd-6097-4524-9c86-44c7f484f278" volumeName="kubernetes.io/empty-dir/c4b9f9bd-6097-4524-9c86-44c7f484f278-data-volume" seLinuxMountContext=""
Oct 30 19:06:08.360624 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.360249    4524 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:06:08.361589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.360260    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d36c6f3a-c76b-4010-898b-9cd567933c5a" volumeName="kubernetes.io/configmap/d36c6f3a-c76b-4010-898b-9cd567933c5a-signing-cabundle" seLinuxMountContext=""
Oct 30 19:06:08.361589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.361439    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="838f8f5e-ac73-4280-8ed3-173a51c42b9e" volumeName="kubernetes.io/projected/838f8f5e-ac73-4280-8ed3-173a51c42b9e-kube-api-access-zpgcn" seLinuxMountContext=""
Oct 30 19:06:08.361589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.361461    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c4b9f9bd-6097-4524-9c86-44c7f484f278" volumeName="kubernetes.io/configmap/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 19:06:08.361589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.361476    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d36c6f3a-c76b-4010-898b-9cd567933c5a" volumeName="kubernetes.io/projected/d36c6f3a-c76b-4010-898b-9cd567933c5a-kube-api-access-vxkwv" seLinuxMountContext=""
Oct 30 19:06:08.361589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.361490    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d36c6f3a-c76b-4010-898b-9cd567933c5a" volumeName="kubernetes.io/secret/d36c6f3a-c76b-4010-898b-9cd567933c5a-signing-key" seLinuxMountContext=""
Oct 30 19:06:08.361589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.361503    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fd24c416-9f3c-4955-a536-08f20ccdbbbb" volumeName="kubernetes.io/projected/fd24c416-9f3c-4955-a536-08f20ccdbbbb-kube-api-access-49bwq" seLinuxMountContext=""
Oct 30 19:06:08.361589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.361515    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1" volumeName="kubernetes.io/empty-dir/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-tmp" seLinuxMountContext=""
Oct 30 19:06:08.361589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.361528    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5fdb2ea2-a56c-46f0-b155-468303552965" volumeName="kubernetes.io/projected/5fdb2ea2-a56c-46f0-b155-468303552965-kube-api-access-qw4hs" seLinuxMountContext=""
Oct 30 19:06:08.361589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.361540    4524 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fd24c416-9f3c-4955-a536-08f20ccdbbbb" volumeName="kubernetes.io/configmap/fd24c416-9f3c-4955-a536-08f20ccdbbbb-iptables-alerter-script" seLinuxMountContext=""
Oct 30 19:06:08.361589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.361553    4524 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 19:06:08.361589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.361561    4524 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 19:06:08.362134 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.362122    4524 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:06:08.362493 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.362211    4524 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:06:08.364691 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.364675    4524 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 19:06:08.380535 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.380506    4524 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 19:06:08.381726 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.381705    4524 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 19:06:08.381726 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.381724    4524 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 19:06:08.381864 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.381743    4524 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 19:06:08.381864 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:06:08.381783    4524 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 19:06:08.391108 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.391085    4524 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 19:06:08.403509 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.403482    4524 generic.go:356] "Generic (PLEG): container finished" podID="b7c1242a-b12d-49b2-8052-781394f01418" containerID="15291a49719c9ee66421e9994d9db770b669de936127bb7d60a87ea1935c5583" exitCode=0
Oct 30 19:06:08.403509 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.403508    4524 generic.go:356] "Generic (PLEG): container finished" podID="b7c1242a-b12d-49b2-8052-781394f01418" containerID="56c27dd8b5c096104c275e78ac2b79f65f2806f0dde5bce846d11c4f2a0ce4fe" exitCode=0
Oct 30 19:06:08.403654 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.403517    4524 generic.go:356] "Generic (PLEG): container finished" podID="b7c1242a-b12d-49b2-8052-781394f01418" containerID="a273df99c005f00ff6a2ec5561ba922b7c712ab10b7cef763ca9b006d6556c38" exitCode=0
Oct 30 19:06:08.403654 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.403528    4524 generic.go:356] "Generic (PLEG): container finished" podID="b7c1242a-b12d-49b2-8052-781394f01418" containerID="35288109388ddbffbde76f0cac03fa3a0d4c11a1eaca766e19ee65b800f68025" exitCode=0
Oct 30 19:06:08.403654 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.403536    4524 generic.go:356] "Generic (PLEG): container finished" podID="b7c1242a-b12d-49b2-8052-781394f01418" containerID="5174e163852289ef2d62bd9c6643e73db8259644771c17b4ebc807fad7359667" exitCode=0
Oct 30 19:06:08.403654 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.403551    4524 generic.go:356] "Generic (PLEG): container finished" podID="b7c1242a-b12d-49b2-8052-781394f01418" containerID="4325ed0d6ea13153968c7f0c587f987f6a6ee5f21ff8886fd8ae09f566134612" exitCode=0
Oct 30 19:06:08.418987 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.418966    4524 generic.go:356] "Generic (PLEG): container finished" podID="7fe3dcaf32fbbd51ccedffe35a8755ab" containerID="a346e3a69fec2fa80723317b10070413f8ab9ee567a5cf9d2f8c9b22f59a303d" exitCode=0
Oct 30 19:06:08.428815 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.428797    4524 manager.go:324] Recovery completed
Oct 30 19:06:08.443754 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.443710    4524 cpu_manager.go:226] "Starting" policy="none"
Oct 30 19:06:08.443754 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.443725    4524 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 19:06:08.443754 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.443744    4524 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 19:06:08.443883 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.443852    4524 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 19:06:08.443883 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.443865    4524 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 19:06:08.443883 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.443882    4524 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 19:06:08.443972 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.443887    4524 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 19:06:08.443972 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.443893    4524 policy_none.go:50] "Start"
Oct 30 19:06:08.443972 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.443901    4524 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 19:06:08.443972 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.443910    4524 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 19:06:08.444081 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.444027    4524 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 19:06:08.444081 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.444034    4524 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 19:06:08.444081 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.444040    4524 policy_none.go:44] "Start"
Oct 30 19:06:08.447988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.447965    4524 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 19:06:08.450598 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.450578    4524 manager.go:341] "Starting Device Plugin manager"
Oct 30 19:06:08.451043 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:06:08.450608    4524 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 19:06:08.451112 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.451046    4524 server.go:87] "Starting device plugin registration server"
Oct 30 19:06:08.451340 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.451327    4524 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 19:06:08.451444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.451341    4524 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 19:06:08.451444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.451423    4524 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 19:06:08.451562 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.451522    4524 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 19:06:08.451615 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.451577    4524 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 19:06:08.451615 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.451586    4524 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 19:06:08.451758 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.451662    4524 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T19:06:08.451560739Z","UUID":"c84b07dc-596c-4624-93d7-035a69b0225e","Handler":null,"Name":"","Endpoint":""}
Oct 30 19:06:08.452945 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:06:08.452927    4524 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 19:06:08.453061 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.453044    4524 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 19:06:08.453107 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.453080    4524 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 19:06:08.482154 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.482107    4524 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal"]
Oct 30 19:06:08.482593 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.482555    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal" event={"ID":"7e14d55fcbb7a4643b4d48f7935b8114","Type":"ContainerStarted","Data":"817e40d960fc2832d270c769b54faddea0c8e02198c1b6a8887c27cc2fce352a"}
Oct 30 19:06:08.482683 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.482631    4524 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:08.482750 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.482681    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal" event={"ID":"7e14d55fcbb7a4643b4d48f7935b8114","Type":"ContainerStarted","Data":"1b6e265adf5cb80c3712e09bc95d489b247261151cb0892a6e6f2e85afb2e715"}
Oct 30 19:06:08.482750 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.482738    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal" event={"ID":"7fe3dcaf32fbbd51ccedffe35a8755ab","Type":"ContainerStarted","Data":"5150f5b5eac32390f7988cd29576fec52fbefb1c90f35f39dd0e2512990d398f"}
Oct 30 19:06:08.482844 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.482754    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal" event={"ID":"7fe3dcaf32fbbd51ccedffe35a8755ab","Type":"ContainerDied","Data":"a346e3a69fec2fa80723317b10070413f8ab9ee567a5cf9d2f8c9b22f59a303d"}
Oct 30 19:06:08.482844 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.482767    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal" event={"ID":"7fe3dcaf32fbbd51ccedffe35a8755ab","Type":"ContainerStarted","Data":"c5816817ad4d94d47e364d6a051169d0f69b695fc2f17e86ec0af6250b70843b"}
Oct 30 19:06:08.482844 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.482768    4524 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:08.495894 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.495875    4524 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:06:08.495956 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:06:08.495923    4524 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:08.496407 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.496394    4524 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:06:08.496442 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:06:08.496427    4524 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:08.551493 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.551472    4524 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 19:06:08.552518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.552502    4524 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 19:06:08.552593 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.552531    4524 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 19:06:08.552593 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.552542    4524 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-137-200.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 19:06:08.552593 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.552570    4524 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:08.565636 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.565618    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7fe3dcaf32fbbd51ccedffe35a8755ab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal\" (UID: \"7fe3dcaf32fbbd51ccedffe35a8755ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:08.565684 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.565643    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7fe3dcaf32fbbd51ccedffe35a8755ab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal\" (UID: \"7fe3dcaf32fbbd51ccedffe35a8755ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:08.565684 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.565658    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7e14d55fcbb7a4643b4d48f7935b8114-config\") pod \"kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal\" (UID: \"7e14d55fcbb7a4643b4d48f7935b8114\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:08.572413 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.572395    4524 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:08.572460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.572438    4524 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:08.666652 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.666628    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7e14d55fcbb7a4643b4d48f7935b8114-config\") pod \"kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal\" (UID: \"7e14d55fcbb7a4643b4d48f7935b8114\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:08.666742 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.666691    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7fe3dcaf32fbbd51ccedffe35a8755ab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal\" (UID: \"7fe3dcaf32fbbd51ccedffe35a8755ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:08.666742 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.666654    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7fe3dcaf32fbbd51ccedffe35a8755ab-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal\" (UID: \"7fe3dcaf32fbbd51ccedffe35a8755ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:08.666810 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.666728    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7e14d55fcbb7a4643b4d48f7935b8114-config\") pod \"kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal\" (UID: \"7e14d55fcbb7a4643b4d48f7935b8114\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:08.666810 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.666752    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7fe3dcaf32fbbd51ccedffe35a8755ab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal\" (UID: \"7fe3dcaf32fbbd51ccedffe35a8755ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:08.666810 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:08.666797    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7fe3dcaf32fbbd51ccedffe35a8755ab-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal\" (UID: \"7fe3dcaf32fbbd51ccedffe35a8755ab\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:09.332398 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.332368    4524 apiserver.go:52] "Watching apiserver"
Oct 30 19:06:09.338399 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.338377    4524 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 19:06:09.338859 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.338837    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-svnhq","openshift-multus/multus-additional-cni-plugins-d2ntn","openshift-network-diagnostics/network-check-target-z2dbh","openshift-ovn-kubernetes/ovnkube-node-2gqlw","openshift-image-registry/node-ca-2z2fd","openshift-insights/periodic-gathering-75z8h-spdtl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal","kube-system/global-pull-secret-syncer-7q5vl","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn","openshift-service-ca/service-ca-5fffc97bdf-5nrmq","openshift-dns/node-resolver-8jrs8","openshift-cluster-node-tuning-operator/tuned-b7bcf","openshift-multus/network-metrics-daemon-xmkt2","kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal","openshift-dns/dns-default-v86zw","openshift-multus/multus-lq697","openshift-network-operator/iptables-alerter-4q28p","openshift-ingress-canary/ingress-canary-xtj4p","kube-system/konnectivity-agent-ln5ql","openshift-insights/insights-runtime-extractor-fht7f"]
Oct 30 19:06:09.344768 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.344741    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 19:06:09.344912 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.344795    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-tc7v2\""
Oct 30 19:06:09.344912 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.344859    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-m74bb\""
Oct 30 19:06:09.344912 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.344901    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 19:06:09.345046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.344939    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hsp67\""
Oct 30 19:06:09.345218 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.345203    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:06:09.346143 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.346127    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v86zw"
Oct 30 19:06:09.347033 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.347008    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 19:06:09.347365 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.347349    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:09.350033 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.349995    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 19:06:09.350033 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.350010    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 19:06:09.350163 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.350045    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 19:06:09.350163 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.350132    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 19:06:09.350163 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.350145    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 19:06:09.352784 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.352772    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xlxzb\""
Oct 30 19:06:09.352969 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.352955    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ngr9b\""
Oct 30 19:06:09.352969 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.352963    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 19:06:09.353073 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.352999    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 19:06:09.353073 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.353008    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-4z6gp\""
Oct 30 19:06:09.353073 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.353020    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-89jvk\""
Oct 30 19:06:09.354997 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.354982    4524 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 19:06:09.369852 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.369798    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:09.369852 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.369828    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-cni-dir\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.369970 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.369854    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-var-lib-cni-bin\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.369970 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.369875    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-hostroot\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.369970 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.369935    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/838f8f5e-ac73-4280-8ed3-173a51c42b9e-host\") pod \"node-ca-2z2fd\" (UID: \"838f8f5e-ac73-4280-8ed3-173a51c42b9e\") " pod="openshift-image-registry/node-ca-2z2fd"
Oct 30 19:06:09.370054 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.369982    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/838f8f5e-ac73-4280-8ed3-173a51c42b9e-serviceca\") pod \"node-ca-2z2fd\" (UID: \"838f8f5e-ac73-4280-8ed3-173a51c42b9e\") " pod="openshift-image-registry/node-ca-2z2fd"
Oct 30 19:06:09.370083 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370070    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-dbus\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:06:09.370111 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370101    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-run-systemd\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.370150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370136    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-cni-bin\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.370304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370182    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:06:09.370304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370208    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-cni-binary-copy\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.370304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370244    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-systemd\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.370304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370268    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-device-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:06:09.370508 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370363    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49bwq\" (UniqueName: \"kubernetes.io/projected/fd24c416-9f3c-4955-a536-08f20ccdbbbb-kube-api-access-49bwq\") pod \"iptables-alerter-4q28p\" (UID: \"fd24c416-9f3c-4955-a536-08f20ccdbbbb\") " pod="openshift-network-operator/iptables-alerter-4q28p"
Oct 30 19:06:09.370508 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370403    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-service-ca-bundle\") pod \"periodic-gathering-75z8h-spdtl\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") " pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:06:09.370508 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370430    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-sysctl-d\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.370508 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370454    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-kubelet\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.370508 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370479    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-run-ovn-kubernetes\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.370659 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370509    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-cni-binary-copy\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.370659 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370567    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-cnibin\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.370659 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370614    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-env-overrides\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.370659 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370635    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd24c416-9f3c-4955-a536-08f20ccdbbbb-host-slash\") pod \"iptables-alerter-4q28p\" (UID: \"fd24c416-9f3c-4955-a536-08f20ccdbbbb\") " pod="openshift-network-operator/iptables-alerter-4q28p"
Oct 30 19:06:09.370771 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370685    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-system-cni-dir\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.370771 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370701    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-var-lib-cni-multus\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.370771 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370716    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7c1242a-b12d-49b2-8052-781394f01418-system-cni-dir\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:06:09.370771 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370743    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-run\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.370771 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370762    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-slash\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.370926 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370784    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-run-netns\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.370926 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370803    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-log-socket\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.370926 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370827    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-registration-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:06:09.370926 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370850    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce07e9b3-5e98-4d16-8158-afba6fe42281-config-volume\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:06:09.370926 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370864    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7411c16b-533b-4152-bbe6-52262afa37da-cert\") pod \"ingress-canary-xtj4p\" (UID: \"7411c16b-533b-4152-bbe6-52262afa37da\") " pod="openshift-ingress-canary/ingress-canary-xtj4p"
Oct 30 19:06:09.370926 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370919    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc2g2\" (UniqueName: \"kubernetes.io/projected/7411c16b-533b-4152-bbe6-52262afa37da-kube-api-access-wc2g2\") pod \"ingress-canary-xtj4p\" (UID: \"7411c16b-533b-4152-bbe6-52262afa37da\") " pod="openshift-ingress-canary/ingress-canary-xtj4p"
Oct 30 19:06:09.371104 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370950    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-etc-openvswitch\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.371104 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.370989    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-etc-selinux\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:06:09.371104 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371011    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:09.371104 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371027    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d36c6f3a-c76b-4010-898b-9cd567933c5a-signing-cabundle\") pod \"service-ca-5fffc97bdf-5nrmq\" (UID: \"d36c6f3a-c76b-4010-898b-9cd567933c5a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-5nrmq"
Oct 30 19:06:09.371104 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371047    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-run-netns\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.371104 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371090    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-conf-dir\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.371388 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371125    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65lrg\" (UniqueName: \"kubernetes.io/projected/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-kube-api-access-65lrg\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.371388 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371197    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-run-ovn\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.371388 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371235    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-var-lib-kubelet\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.371388 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371274    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-var-lib-openvswitch\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.371388 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371311    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-socket-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:06:09.371388 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371335    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxkwv\" (UniqueName: \"kubernetes.io/projected/d36c6f3a-c76b-4010-898b-9cd567933c5a-kube-api-access-vxkwv\") pod \"service-ca-5fffc97bdf-5nrmq\" (UID: \"d36c6f3a-c76b-4010-898b-9cd567933c5a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-5nrmq"
Oct 30 19:06:09.371690 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371391    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-run-k8s-cni-cncf-io\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.371690 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371419    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-run-multus-certs\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.371690 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371447    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pq2q\" (UniqueName: \"kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q\") pod \"network-check-target-z2dbh\" (UID: \"c1cc12f1-be52-4a9f-a8a2-53fe3c493431\") " pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:06:09.371690 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371485    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-cni-netd\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.371690 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371609    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-ovnkube-script-lib\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.371690 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371627    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/94709da2-704a-421c-aa2f-1844db5bec96-hosts-file\") pod \"node-resolver-8jrs8\" (UID: \"94709da2-704a-421c-aa2f-1844db5bec96\") " pod="openshift-dns/node-resolver-8jrs8"
Oct 30 19:06:09.371690 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371641    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-var-lib-kubelet\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.371690 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371658    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8d23c4bb-0012-4e09-bc59-c6689f9ea394-agent-certs\") pod \"konnectivity-agent-ln5ql\" (UID: \"8d23c4bb-0012-4e09-bc59-c6689f9ea394\") " pod="kube-system/konnectivity-agent-ln5ql"
Oct 30 19:06:09.371690 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371680    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7c1242a-b12d-49b2-8052-781394f01418-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:06:09.372041 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371752    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs\") pod \"network-metrics-daemon-xmkt2\" (UID: \"ed4e7b61-d816-49fc-b4d9-36236a716227\") " pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:06:09.372041 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371823    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-kubelet-config\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:06:09.372041 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371863    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw4hs\" (UniqueName: \"kubernetes.io/projected/5fdb2ea2-a56c-46f0-b155-468303552965-kube-api-access-qw4hs\") pod \"migrator-66df5d496b-svnhq\" (UID: \"5fdb2ea2-a56c-46f0-b155-468303552965\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-svnhq"
Oct 30 19:06:09.372041 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371899    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fd24c416-9f3c-4955-a536-08f20ccdbbbb-iptables-alerter-script\") pod \"iptables-alerter-4q28p\" (UID: \"fd24c416-9f3c-4955-a536-08f20ccdbbbb\") " pod="openshift-network-operator/iptables-alerter-4q28p"
Oct 30 19:06:09.372041 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371928    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8d23c4bb-0012-4e09-bc59-c6689f9ea394-agent-certs\") pod \"konnectivity-agent-ln5ql\" (UID: \"8d23c4bb-0012-4e09-bc59-c6689f9ea394\") " pod="kube-system/konnectivity-agent-ln5ql"
Oct 30 19:06:09.372041 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371939    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d36c6f3a-c76b-4010-898b-9cd567933c5a-signing-key\") pod \"service-ca-5fffc97bdf-5nrmq\" (UID: \"d36c6f3a-c76b-4010-898b-9cd567933c5a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-5nrmq"
Oct 30 19:06:09.372041 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371973    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-socket-dir-parent\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.372041 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.371998    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-etc-kubernetes\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.372463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372053    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7c1242a-b12d-49b2-8052-781394f01418-cnibin\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:06:09.372463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372076    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqgn5\" (UniqueName: \"kubernetes.io/projected/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-kube-api-access-wqgn5\") pod \"periodic-gathering-75z8h-spdtl\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") " pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:06:09.372463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372092    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-lib-modules\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.372463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372108    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jmzz\" (UniqueName: \"kubernetes.io/projected/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-api-access-6jmzz\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:09.372463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372146    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-daemon-config\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.372463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372209    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7c1242a-b12d-49b2-8052-781394f01418-cni-binary-copy\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:06:09.372463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372241    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4hr8\" (UniqueName: \"kubernetes.io/projected/ed4e7b61-d816-49fc-b4d9-36236a716227-kube-api-access-b4hr8\") pod \"network-metrics-daemon-xmkt2\" (UID: \"ed4e7b61-d816-49fc-b4d9-36236a716227\") " pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:06:09.372463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372267    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-sysconfig\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.372463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372292    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-tuned\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.372463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372330    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-daemon-config\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.372463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372337    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-tmp\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.372463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372365    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hf6x\" (UniqueName: \"kubernetes.io/projected/94709da2-704a-421c-aa2f-1844db5bec96-kube-api-access-6hf6x\") pod \"node-resolver-8jrs8\" (UID: \"94709da2-704a-421c-aa2f-1844db5bec96\") " pod="openshift-dns/node-resolver-8jrs8"
Oct 30 19:06:09.372463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372384    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8d23c4bb-0012-4e09-bc59-c6689f9ea394-konnectivity-ca\") pod \"konnectivity-agent-ln5ql\" (UID: \"8d23c4bb-0012-4e09-bc59-c6689f9ea394\") " pod="kube-system/konnectivity-agent-ln5ql"
Oct 30 19:06:09.372463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372405    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:06:09.372463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372413    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-tmp\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.372463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372418    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-tuned\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.372463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372445    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b7c1242a-b12d-49b2-8052-781394f01418-cni-binary-copy\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372422    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-kubernetes\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372513    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-sysctl-conf\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372528    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-sys\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372594    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-run-openvswitch\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372624    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-node-log\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372649    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372679    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxvt7\" (UniqueName: \"kubernetes.io/projected/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-kube-api-access-qxvt7\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372705    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c4b9f9bd-6097-4524-9c86-44c7f484f278-crio-socket\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372758    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-modprobe-d\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372807    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-host\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372836    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-ovnkube-config\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372861    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-ovn-node-metrics-cert\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372886    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nlrh\" (UniqueName: \"kubernetes.io/projected/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-kube-api-access-9nlrh\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372912    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce07e9b3-5e98-4d16-8158-afba6fe42281-metrics-tls\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.372990    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94709da2-704a-421c-aa2f-1844db5bec96-tmp-dir\") pod \"node-resolver-8jrs8\" (UID: \"94709da2-704a-421c-aa2f-1844db5bec96\") " pod="openshift-dns/node-resolver-8jrs8"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.373028    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4b9f9bd-6097-4524-9c86-44c7f484f278-data-volume\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.373056    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdsdn\" (UniqueName: \"kubernetes.io/projected/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-kube-api-access-qdsdn\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.373085    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94709da2-704a-421c-aa2f-1844db5bec96-tmp-dir\") pod \"node-resolver-8jrs8\" (UID: \"94709da2-704a-421c-aa2f-1844db5bec96\") " pod="openshift-dns/node-resolver-8jrs8"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.373137    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce07e9b3-5e98-4d16-8158-afba6fe42281-tmp-dir\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.373163    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxhpm\" (UniqueName: \"kubernetes.io/projected/ce07e9b3-5e98-4d16-8158-afba6fe42281-kube-api-access-wxhpm\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.373139    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4b9f9bd-6097-4524-9c86-44c7f484f278-data-volume\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.373202    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7c1242a-b12d-49b2-8052-781394f01418-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.373206    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce07e9b3-5e98-4d16-8158-afba6fe42281-tmp-dir\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:06:09.373239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.373239    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvx2d\" (UniqueName: \"kubernetes.io/projected/b7c1242a-b12d-49b2-8052-781394f01418-kube-api-access-qvx2d\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:06:09.374023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.373267    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-archives-path\") pod \"periodic-gathering-75z8h-spdtl\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") " pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:06:09.374023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.373272    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-ovn-node-metrics-cert\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.374023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.373363    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-trusted-ca-bundle\") pod \"periodic-gathering-75z8h-spdtl\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") " pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:06:09.374023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.373403    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-systemd-units\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.374023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.373434    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-sys-fs\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:06:09.374023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.373478    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7c1242a-b12d-49b2-8052-781394f01418-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:06:09.374023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.373500    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpgcn\" (UniqueName: \"kubernetes.io/projected/838f8f5e-ac73-4280-8ed3-173a51c42b9e-kube-api-access-zpgcn\") pod \"node-ca-2z2fd\" (UID: \"838f8f5e-ac73-4280-8ed3-173a51c42b9e\") " pod="openshift-image-registry/node-ca-2z2fd"
Oct 30 19:06:09.374023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.373584    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-archives-path\") pod \"periodic-gathering-75z8h-spdtl\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") " pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:06:09.387034 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.387015    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hf6x\" (UniqueName: \"kubernetes.io/projected/94709da2-704a-421c-aa2f-1844db5bec96-kube-api-access-6hf6x\") pod \"node-resolver-8jrs8\" (UID: \"94709da2-704a-421c-aa2f-1844db5bec96\") " pod="openshift-dns/node-resolver-8jrs8"
Oct 30 19:06:09.390667 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.390649    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvx2d\" (UniqueName: \"kubernetes.io/projected/b7c1242a-b12d-49b2-8052-781394f01418-kube-api-access-qvx2d\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:06:09.392269 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.392254    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pq2q\" (UniqueName: \"kubernetes.io/projected/c1cc12f1-be52-4a9f-a8a2-53fe3c493431-kube-api-access-8pq2q\") pod \"network-check-target-z2dbh\" (UID: \"c1cc12f1-be52-4a9f-a8a2-53fe3c493431\") " pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:06:09.392442 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.392427    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4hr8\" (UniqueName: \"kubernetes.io/projected/ed4e7b61-d816-49fc-b4d9-36236a716227-kube-api-access-b4hr8\") pod \"network-metrics-daemon-xmkt2\" (UID: \"ed4e7b61-d816-49fc-b4d9-36236a716227\") " pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:06:09.398379 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.398361    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 19:06:09.398559 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.398545    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 19:06:09.398707 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.398695    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 19:06:09.399953 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.399940    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 19:06:09.400201 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400187    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:06:09.400250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400238    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:06:09.400289 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400272    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/838f8f5e-ac73-4280-8ed3-173a51c42b9e-serviceca\") pod \"node-ca-2z2fd\" (UID: \"838f8f5e-ac73-4280-8ed3-173a51c42b9e\") " pod="openshift-image-registry/node-ca-2z2fd"
Oct 30 19:06:09.400328 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400288    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 19:06:09.400328 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400319    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 19:06:09.400407 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400395    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 19:06:09.400443 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400406    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 19:06:09.400443 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400433    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 19:06:09.400513 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400444    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 19:06:09.400513 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400479    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 19:06:09.400513 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400491    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 19:06:09.400602 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400516    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 19:06:09.400602 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400530    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Oct 30 19:06:09.400602 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400539    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 19:06:09.400602 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400569    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 19:06:09.400602 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400575    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Oct 30 19:06:09.400748 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400607    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 19:06:09.400748 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400573    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Oct 30 19:06:09.400748 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400628    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Oct 30 19:06:09.400748 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400633    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 19:06:09.400748 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400667    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 19:06:09.400748 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400676    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Oct 30 19:06:09.400748 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400686    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 19:06:09.400748 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400697    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Oct 30 19:06:09.400748 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400736    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 19:06:09.401062 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400851    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-env-overrides\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.401062 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.400872    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 19:06:09.401208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.401189    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxvt7\" (UniqueName: \"kubernetes.io/projected/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-kube-api-access-qxvt7\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.401283 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.401244    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:09.401364 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.401327    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d36c6f3a-c76b-4010-898b-9cd567933c5a-signing-cabundle\") pod \"service-ca-5fffc97bdf-5nrmq\" (UID: \"d36c6f3a-c76b-4010-898b-9cd567933c5a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-5nrmq"
Oct 30 19:06:09.401427 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.401353    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-service-ca-bundle\") pod \"periodic-gathering-75z8h-spdtl\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") " pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:06:09.401427 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.401382    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 19:06:09.401516 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.401466    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce07e9b3-5e98-4d16-8158-afba6fe42281-config-volume\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:06:09.401724 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.401706    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpgcn\" (UniqueName: \"kubernetes.io/projected/838f8f5e-ac73-4280-8ed3-173a51c42b9e-kube-api-access-zpgcn\") pod \"node-ca-2z2fd\" (UID: \"838f8f5e-ac73-4280-8ed3-173a51c42b9e\") " pod="openshift-image-registry/node-ca-2z2fd"
Oct 30 19:06:09.401847 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.401830    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nlrh\" (UniqueName: \"kubernetes.io/projected/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-kube-api-access-9nlrh\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:06:09.401932 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.401900    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-ovnkube-script-lib\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.401932 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.401915    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b7c1242a-b12d-49b2-8052-781394f01418-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:06:09.402113 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.402096    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw4hs\" (UniqueName: \"kubernetes.io/projected/5fdb2ea2-a56c-46f0-b155-468303552965-kube-api-access-qw4hs\") pod \"migrator-66df5d496b-svnhq\" (UID: \"5fdb2ea2-a56c-46f0-b155-468303552965\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-svnhq"
Oct 30 19:06:09.402196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.402119    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fd24c416-9f3c-4955-a536-08f20ccdbbbb-iptables-alerter-script\") pod \"iptables-alerter-4q28p\" (UID: \"fd24c416-9f3c-4955-a536-08f20ccdbbbb\") " pod="openshift-network-operator/iptables-alerter-4q28p"
Oct 30 19:06:09.402879 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.402863    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8d23c4bb-0012-4e09-bc59-c6689f9ea394-konnectivity-ca\") pod \"konnectivity-agent-ln5ql\" (UID: \"8d23c4bb-0012-4e09-bc59-c6689f9ea394\") " pod="kube-system/konnectivity-agent-ln5ql"
Oct 30 19:06:09.403323 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.403067    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-ovnkube-config\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.404105 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.404064    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Oct 30 19:06:09.404266 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.404244    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b7c1242a-b12d-49b2-8052-781394f01418-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:06:09.406188 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.406133    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdsdn\" (UniqueName: \"kubernetes.io/projected/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-kube-api-access-qdsdn\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.406928 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.406910    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxhpm\" (UniqueName: \"kubernetes.io/projected/ce07e9b3-5e98-4d16-8158-afba6fe42281-kube-api-access-wxhpm\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:06:09.406983 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.406967    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65lrg\" (UniqueName: \"kubernetes.io/projected/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-kube-api-access-65lrg\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.407257 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.407243    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jmzz\" (UniqueName: \"kubernetes.io/projected/c4b9f9bd-6097-4524-9c86-44c7f484f278-kube-api-access-6jmzz\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:09.409962 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.409939    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49bwq\" (UniqueName: \"kubernetes.io/projected/fd24c416-9f3c-4955-a536-08f20ccdbbbb-kube-api-access-49bwq\") pod \"iptables-alerter-4q28p\" (UID: \"fd24c416-9f3c-4955-a536-08f20ccdbbbb\") " pod="openshift-network-operator/iptables-alerter-4q28p"
Oct 30 19:06:09.410075 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.409991    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqgn5\" (UniqueName: \"kubernetes.io/projected/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-kube-api-access-wqgn5\") pod \"periodic-gathering-75z8h-spdtl\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") " pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:06:09.410075 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.410052    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxkwv\" (UniqueName: \"kubernetes.io/projected/d36c6f3a-c76b-4010-898b-9cd567933c5a-kube-api-access-vxkwv\") pod \"service-ca-5fffc97bdf-5nrmq\" (UID: \"d36c6f3a-c76b-4010-898b-9cd567933c5a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-5nrmq"
Oct 30 19:06:09.410151 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.410104    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc2g2\" (UniqueName: \"kubernetes.io/projected/7411c16b-533b-4152-bbe6-52262afa37da-kube-api-access-wc2g2\") pod \"ingress-canary-xtj4p\" (UID: \"7411c16b-533b-4152-bbe6-52262afa37da\") " pod="openshift-ingress-canary/ingress-canary-xtj4p"
Oct 30 19:06:09.414088 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.414072    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-trusted-ca-bundle\") pod \"periodic-gathering-75z8h-spdtl\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") " pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:06:09.421158 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.421141    4524 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:09.421312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.421296    4524 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:09.433224 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.433209    4524 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:06:09.433293 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:06:09.433245    4524 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:09.436312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.436297    4524 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 19:06:09.436357 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:06:09.436337    4524 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-200.us-east-2.compute.internal"
Oct 30 19:06:09.446090 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.446074    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 19:06:09.446185 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.446135    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 19:06:09.446236 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.446204    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 19:06:09.446236 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.446217    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Oct 30 19:06:09.446304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.446204    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-9klsx\""
Oct 30 19:06:09.446304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.446246    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-w2h58\""
Oct 30 19:06:09.446304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.446246    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 19:06:09.446396 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.446349    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 19:06:09.446634 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.446618    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6h85f\""
Oct 30 19:06:09.446679 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.446619    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-6qlhx\""
Oct 30 19:06:09.446679 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.446627    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-fsc2n\""
Oct 30 19:06:09.446816 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.446802    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-xxr6f\""
Oct 30 19:06:09.446850 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.446834    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 19:06:09.446895 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.446884    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gkzrx\""
Oct 30 19:06:09.447006 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.446994    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-stgll\""
Oct 30 19:06:09.447110 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.447099    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xd9sf\""
Oct 30 19:06:09.447150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.447119    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jks97\""
Oct 30 19:06:09.447150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.447138    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fljvz\""
Oct 30 19:06:09.451300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.451284    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7411c16b-533b-4152-bbe6-52262afa37da-cert\") pod \"ingress-canary-xtj4p\" (UID: \"7411c16b-533b-4152-bbe6-52262afa37da\") " pod="openshift-ingress-canary/ingress-canary-xtj4p"
Oct 30 19:06:09.452388 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.452373    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d36c6f3a-c76b-4010-898b-9cd567933c5a-signing-key\") pod \"service-ca-5fffc97bdf-5nrmq\" (UID: \"d36c6f3a-c76b-4010-898b-9cd567933c5a\") " pod="openshift-service-ca/service-ca-5fffc97bdf-5nrmq"
Oct 30 19:06:09.453576 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.453562    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-original-pull-secret\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:06:09.473859 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.473842    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/838f8f5e-ac73-4280-8ed3-173a51c42b9e-host\") pod \"node-ca-2z2fd\" (UID: \"838f8f5e-ac73-4280-8ed3-173a51c42b9e\") " pod="openshift-image-registry/node-ca-2z2fd"
Oct 30 19:06:09.473932 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.473904    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-dbus\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:06:09.473976 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.473949    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/838f8f5e-ac73-4280-8ed3-173a51c42b9e-host\") pod \"node-ca-2z2fd\" (UID: \"838f8f5e-ac73-4280-8ed3-173a51c42b9e\") " pod="openshift-image-registry/node-ca-2z2fd"
Oct 30 19:06:09.474018 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.473972    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-dbus\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:06:09.474018 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.473997    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-run-systemd\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.474075 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474023    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-cni-bin\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.474075 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474046    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:06:09.474128 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474093    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-systemd\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.474128 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474115    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:06:09.474214 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474126    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-device-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:06:09.474214 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474113    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-cni-bin\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.474214 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474155    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-sysctl-d\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.474214 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474161    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-systemd\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.474214 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474191    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-run-systemd\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.474214 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474201    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-kubelet\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.474214 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474204    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-device-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:06:09.474444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474225    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-run-ovn-kubernetes\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.474444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474249    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-cnibin\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.474444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474249    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-kubelet\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.474444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474273    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd24c416-9f3c-4955-a536-08f20ccdbbbb-host-slash\") pod \"iptables-alerter-4q28p\" (UID: \"fd24c416-9f3c-4955-a536-08f20ccdbbbb\") " pod="openshift-network-operator/iptables-alerter-4q28p"
Oct 30 19:06:09.474444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474294    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-run-ovn-kubernetes\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.474444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474297    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-system-cni-dir\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.474444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474315    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-sysctl-d\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.474444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474328    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fd24c416-9f3c-4955-a536-08f20ccdbbbb-host-slash\") pod \"iptables-alerter-4q28p\" (UID: \"fd24c416-9f3c-4955-a536-08f20ccdbbbb\") " pod="openshift-network-operator/iptables-alerter-4q28p"
Oct 30 19:06:09.474444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474305    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-cnibin\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.474444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474329    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-var-lib-cni-multus\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.474444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474343    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-system-cni-dir\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.474444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474359    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-var-lib-cni-multus\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.474444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474361    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7c1242a-b12d-49b2-8052-781394f01418-system-cni-dir\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:06:09.474444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474385    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-run\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.474444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474401    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-slash\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.474444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474386    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b7c1242a-b12d-49b2-8052-781394f01418-system-cni-dir\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:06:09.474444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474417    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-run-netns\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.474444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474439    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-run\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.474444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474446    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-run-netns\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474458    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-log-socket\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474480    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-registration-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474479    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-slash\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474499    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-etc-openvswitch\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474513    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-etc-selinux\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474554    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-etc-selinux\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474560    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-log-socket\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474573    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-run-netns\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474598    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-conf-dir\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474554    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-registration-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474614    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-run-ovn\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474624    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-run-netns\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474556    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-etc-openvswitch\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474629    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-var-lib-kubelet\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474654    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-conf-dir\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474662    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-run-ovn\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474659    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-var-lib-openvswitch\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474677    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-var-lib-openvswitch\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474696    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-socket-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474670    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-var-lib-kubelet\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474722    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-run-k8s-cni-cncf-io\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474749    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-run-k8s-cni-cncf-io\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474756    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-socket-dir\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474750    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-run-multus-certs\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474783    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-run-multus-certs\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474785    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-cni-netd\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474802    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-cni-netd\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474819    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/94709da2-704a-421c-aa2f-1844db5bec96-hosts-file\") pod \"node-resolver-8jrs8\" (UID: \"94709da2-704a-421c-aa2f-1844db5bec96\") " pod="openshift-dns/node-resolver-8jrs8"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474844    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-var-lib-kubelet\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474868    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs\") pod \"network-metrics-daemon-xmkt2\" (UID: \"ed4e7b61-d816-49fc-b4d9-36236a716227\") " pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474876    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/94709da2-704a-421c-aa2f-1844db5bec96-hosts-file\") pod \"node-resolver-8jrs8\" (UID: \"94709da2-704a-421c-aa2f-1844db5bec96\") " pod="openshift-dns/node-resolver-8jrs8"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474892    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-kubelet-config\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474904    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-var-lib-kubelet\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474935    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-socket-dir-parent\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474958    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-etc-kubernetes\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474991    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ba82c577-b89d-44f7-87e4-2d00d1e2a5de-kubelet-config\") pod \"global-pull-secret-syncer-7q5vl\" (UID: \"ba82c577-b89d-44f7-87e4-2d00d1e2a5de\") " pod="kube-system/global-pull-secret-syncer-7q5vl"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.474981    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7c1242a-b12d-49b2-8052-781394f01418-cnibin\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475009    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-socket-dir-parent\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475025    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-lib-modules\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475030    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-etc-kubernetes\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475046    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-sysconfig\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475034    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b7c1242a-b12d-49b2-8052-781394f01418-cnibin\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475085    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-kubernetes\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475102    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-sysconfig\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475109    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-sysctl-conf\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475120    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-kubernetes\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475132    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-sys\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475138    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-lib-modules\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475158    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-run-openvswitch\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475183    4524 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475194    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-node-log\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475212    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-run-openvswitch\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475220    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475247    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-node-log\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475267    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-sysctl-conf\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475271    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c4b9f9bd-6097-4524-9c86-44c7f484f278-crio-socket\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475310    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-modprobe-d\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475316    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c4b9f9bd-6097-4524-9c86-44c7f484f278-crio-socket\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475188    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-sys\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.475287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475331    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-host\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.477370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475365    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce07e9b3-5e98-4d16-8158-afba6fe42281-metrics-tls\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:06:09.477370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475371    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-host\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.477370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475284    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.477370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475393    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7c1242a-b12d-49b2-8052-781394f01418-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:06:09.477370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475396    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1-etc-modprobe-d\") pod \"tuned-b7bcf\" (UID: \"fef7bdd6-e0b1-4a2c-ab40-274408d6d0d1\") " pod="openshift-cluster-node-tuning-operator/tuned-b7bcf"
Oct 30 19:06:09.477370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475443    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-systemd-units\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.477370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475445    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b7c1242a-b12d-49b2-8052-781394f01418-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d2ntn\" (UID: \"b7c1242a-b12d-49b2-8052-781394f01418\") " pod="openshift-multus/multus-additional-cni-plugins-d2ntn"
Oct 30 19:06:09.477370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475475    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-sys-fs\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:06:09.477370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475479    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/30fc2aa0-31d5-4577-8064-25dbb6cea5d7-systemd-units\") pod \"ovnkube-node-2gqlw\" (UID: \"30fc2aa0-31d5-4577-8064-25dbb6cea5d7\") " pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:09.477370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475503    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:09.477370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475529    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-cni-dir\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.477370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475539    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/260d4781-1e14-4a9a-8840-49c1f9bdbfbe-sys-fs\") pod \"aws-ebs-csi-driver-node-4w9nn\" (UID: \"260d4781-1e14-4a9a-8840-49c1f9bdbfbe\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4w9nn"
Oct 30 19:06:09.477370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475552    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-var-lib-cni-bin\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.477370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475575    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-hostroot\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.477370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475599    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-multus-cni-dir\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.477370 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:06:09.475613    4524 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 19:06:09.477370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475637    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-host-var-lib-cni-bin\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.477370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.475649    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a-hostroot\") pod \"multus-lq697\" (UID: \"cbcf7627-e9b7-49ee-80ff-d2f05c8e9d3a\") " pod="openshift-multus/multus-lq697"
Oct 30 19:06:09.477370 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:06:09.475736    4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls podName:c4b9f9bd-6097-4524-9c86-44c7f484f278 nodeName:}" failed. No retries permitted until 2026-10-30 19:06:09.975714797 +0000 UTC m=+1.716705871 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fht7f" (UID: "c4b9f9bd-6097-4524-9c86-44c7f484f278") : secret "insights-runtime-extractor-tls" not found
Oct 30 19:06:09.477370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.477324    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed4e7b61-d816-49fc-b4d9-36236a716227-metrics-certs\") pod \"network-metrics-daemon-xmkt2\" (UID: \"ed4e7b61-d816-49fc-b4d9-36236a716227\") " pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:06:09.478313 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.477565    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce07e9b3-5e98-4d16-8158-afba6fe42281-metrics-tls\") pod \"dns-default-v86zw\" (UID: \"ce07e9b3-5e98-4d16-8158-afba6fe42281\") " pod="openshift-dns/dns-default-v86zw"
Oct 30 19:06:09.645496 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.645443    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xmkt2"
Oct 30 19:06:09.646452 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.646438    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-v86zw"
Oct 30 19:06:09.950096 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.949983    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xmkt2"]
Oct 30 19:06:09.953968 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:09.953936    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded4e7b61_d816_49fc_b4d9_36236a716227.slice/crio-0eddd5b75293dab0bb3e906268561eeb3797d54b041bf5d7dcba5bdc412bb50d WatchSource:0}: Error finding container 0eddd5b75293dab0bb3e906268561eeb3797d54b041bf5d7dcba5bdc412bb50d: Status 404 returned error can't find the container with id 0eddd5b75293dab0bb3e906268561eeb3797d54b041bf5d7dcba5bdc412bb50d
Oct 30 19:06:09.956323 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.956306    4524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:06:09.977987 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.977956    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-v86zw"]
Oct 30 19:06:09.979671 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:09.979609    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:09.979726 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:06:09.979710    4524 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 19:06:09.979792 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:06:09.979753    4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls podName:c4b9f9bd-6097-4524-9c86-44c7f484f278 nodeName:}" failed. No retries permitted until 2026-10-30 19:06:10.979741517 +0000 UTC m=+2.720732587 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fht7f" (UID: "c4b9f9bd-6097-4524-9c86-44c7f484f278") : secret "insights-runtime-extractor-tls" not found
Oct 30 19:06:09.981443 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:09.981423    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce07e9b3_5e98_4d16_8158_afba6fe42281.slice/crio-a52e58e7bb06d14ed7b48e019e4be1f867c3691c1adf0d81d15e4d6c9ff23fb8 WatchSource:0}: Error finding container a52e58e7bb06d14ed7b48e019e4be1f867c3691c1adf0d81d15e4d6c9ff23fb8: Status 404 returned error can't find the container with id a52e58e7bb06d14ed7b48e019e4be1f867c3691c1adf0d81d15e4d6c9ff23fb8
Oct 30 19:06:10.423940 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:10.423896    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v86zw" event={"ID":"ce07e9b3-5e98-4d16-8158-afba6fe42281","Type":"ContainerStarted","Data":"a52e58e7bb06d14ed7b48e019e4be1f867c3691c1adf0d81d15e4d6c9ff23fb8"}
Oct 30 19:06:10.425018 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:10.424995    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xmkt2" event={"ID":"ed4e7b61-d816-49fc-b4d9-36236a716227","Type":"ContainerStarted","Data":"0eddd5b75293dab0bb3e906268561eeb3797d54b041bf5d7dcba5bdc412bb50d"}
Oct 30 19:06:10.986936 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:10.986897    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:10.987093 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:06:10.987045    4524 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 19:06:10.987144 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:06:10.987113    4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls podName:c4b9f9bd-6097-4524-9c86-44c7f484f278 nodeName:}" failed. No retries permitted until 2026-10-30 19:06:12.987095605 +0000 UTC m=+4.728086676 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fht7f" (UID: "c4b9f9bd-6097-4524-9c86-44c7f484f278") : secret "insights-runtime-extractor-tls" not found
Oct 30 19:06:11.052578 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:11.052544    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:11.431319 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:11.431278    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xmkt2" event={"ID":"ed4e7b61-d816-49fc-b4d9-36236a716227","Type":"ContainerStarted","Data":"27e27531147327882e117e4b196b0f8e2ff0ea10c3297064d98502fd207dfb6c"}
Oct 30 19:06:11.431786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:11.431337    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xmkt2" event={"ID":"ed4e7b61-d816-49fc-b4d9-36236a716227","Type":"ContainerStarted","Data":"3cbbd6ccfb6155a480308684587542e130a5040185a1a96c29e39d53b318794c"}
Oct 30 19:06:12.440036 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:12.440002    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v86zw" event={"ID":"ce07e9b3-5e98-4d16-8158-afba6fe42281","Type":"ContainerStarted","Data":"5c2038e28bffb5ab32a9704d4ca92b7a6ee7c7a6487e0cee2a5e254e572e0453"}
Oct 30 19:06:12.440443 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:12.440075    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-v86zw" event={"ID":"ce07e9b3-5e98-4d16-8158-afba6fe42281","Type":"ContainerStarted","Data":"761c64c7d80791d9404e4c143d45ba0e432c5c0093181173b3c87f9754012dea"}
Oct 30 19:06:13.002087 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:13.002058    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:13.002255 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:06:13.002166    4524 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Oct 30 19:06:13.002255 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:06:13.002229    4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls podName:c4b9f9bd-6097-4524-9c86-44c7f484f278 nodeName:}" failed. No retries permitted until 2026-10-30 19:06:17.0022144 +0000 UTC m=+8.743205457 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fht7f" (UID: "c4b9f9bd-6097-4524-9c86-44c7f484f278") : secret "insights-runtime-extractor-tls" not found
Oct 30 19:06:13.443447 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:13.443424    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-v86zw"
Oct 30 19:06:16.466064 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:16.466022    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:16.466527 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:16.466260    4524 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 19:06:17.022308 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:17.022278    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:17.024556 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:17.024536    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4b9f9bd-6097-4524-9c86-44c7f484f278-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fht7f\" (UID: \"c4b9f9bd-6097-4524-9c86-44c7f484f278\") " pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:17.147829 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:17.147794    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fht7f"
Oct 30 19:06:17.418140 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:17.418110    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fht7f"]
Oct 30 19:06:17.456371 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:17.456342    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fht7f" event={"ID":"c4b9f9bd-6097-4524-9c86-44c7f484f278","Type":"ContainerStarted","Data":"e2bd9dd4ca3aa85e85f49abf99b261b6925cf8d1826884096c490f47c47d8324"}
Oct 30 19:06:17.953251 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:17.953216    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:06:18.460321 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:18.460279    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fht7f" event={"ID":"c4b9f9bd-6097-4524-9c86-44c7f484f278","Type":"ContainerStarted","Data":"680809d07c8c5f6cb0e786db39a4211f064d050c73129e2cb0a0d8cd7f2d6962"}
Oct 30 19:06:18.460571 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:18.460339    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fht7f" event={"ID":"c4b9f9bd-6097-4524-9c86-44c7f484f278","Type":"ContainerStarted","Data":"04ebb15862b956379a3ed73fd5b7b4d6c1b0de1db6296015de3fd2ce7fc4ee0f"}
Oct 30 19:06:18.684603 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:18.684576    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2gqlw"
Oct 30 19:06:20.467307 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:20.467270    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fht7f" event={"ID":"c4b9f9bd-6097-4524-9c86-44c7f484f278","Type":"ContainerStarted","Data":"bc79d64af53804dd79417fb95750c3722936327ef615ba8c983e06acc5a23176"}
Oct 30 19:06:21.998345 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:21.998314    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-zs5lf-ptq4h"]
Oct 30 19:06:22.000515 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.000496    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-zs5lf-ptq4h"
Oct 30 19:06:22.020489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.020461    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-zs5lf-ptq4h"]
Oct 30 19:06:22.021142 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.021117    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-ztp7b"]
Oct 30 19:06:22.026409 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.026389    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-ztp7b"
Oct 30 19:06:22.031343 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.031321    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-v6lqx"]
Oct 30 19:06:22.034077 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.034060    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-v6lqx"
Oct 30 19:06:22.036496 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.036443    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-ztp7b"]
Oct 30 19:06:22.038904 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.038884    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-64dr2\""
Oct 30 19:06:22.039042 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.039019    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Oct 30 19:06:22.050896 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.050873    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 19:06:22.050999 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.050883    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 19:06:22.052815 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.052786    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpr29\" (UniqueName: \"kubernetes.io/projected/579247bb-977d-4c89-99cd-2831cb2ed824-kube-api-access-gpr29\") pod \"periodic-gathering-zs5lf-ptq4h\" (UID: \"579247bb-977d-4c89-99cd-2831cb2ed824\") " pod="openshift-insights/periodic-gathering-zs5lf-ptq4h"
Oct 30 19:06:22.053258 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.052834    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/579247bb-977d-4c89-99cd-2831cb2ed824-archives-path\") pod \"periodic-gathering-zs5lf-ptq4h\" (UID: \"579247bb-977d-4c89-99cd-2831cb2ed824\") " pod="openshift-insights/periodic-gathering-zs5lf-ptq4h"
Oct 30 19:06:22.053258 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.052896    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/579247bb-977d-4c89-99cd-2831cb2ed824-service-ca-bundle\") pod \"periodic-gathering-zs5lf-ptq4h\" (UID: \"579247bb-977d-4c89-99cd-2831cb2ed824\") " pod="openshift-insights/periodic-gathering-zs5lf-ptq4h"
Oct 30 19:06:22.053258 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.052932    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/579247bb-977d-4c89-99cd-2831cb2ed824-trusted-ca-bundle\") pod \"periodic-gathering-zs5lf-ptq4h\" (UID: \"579247bb-977d-4c89-99cd-2831cb2ed824\") " pod="openshift-insights/periodic-gathering-zs5lf-ptq4h"
Oct 30 19:06:22.053258 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.053226    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-v6lqx"]
Oct 30 19:06:22.116516 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.116486    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b4449ff4b-jmtvm"]
Oct 30 19:06:22.118864 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.118842    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.126266 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.126246    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Oct 30 19:06:22.133911 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.133889    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b4449ff4b-jmtvm"]
Oct 30 19:06:22.137391 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.137367    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-p5tql\""
Oct 30 19:06:22.137492 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.137432    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Oct 30 19:06:22.138198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.137612    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Oct 30 19:06:22.138198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.137722    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vt4hq\""
Oct 30 19:06:22.138198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.137749    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Oct 30 19:06:22.153390 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.153368    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxj9z\" (UniqueName: \"kubernetes.io/projected/9f98e875-33c4-473e-889b-223453fdd51b-kube-api-access-hxj9z\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.153390 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.153395    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8f4e6b19-4c02-43e5-ac9b-fb80af56e5f0-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-ztp7b\" (UID: \"8f4e6b19-4c02-43e5-ac9b-fb80af56e5f0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-ztp7b"
Oct 30 19:06:22.153531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.153418    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f98e875-33c4-473e-889b-223453fdd51b-trusted-ca\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.153531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.153502    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e25a61d-8a82-40ba-aad3-81ad9f762df7-tmp\") pod \"downloads-6dd789c8b7-v6lqx\" (UID: \"5e25a61d-8a82-40ba-aad3-81ad9f762df7\") " pod="openshift-console/downloads-6dd789c8b7-v6lqx"
Oct 30 19:06:22.153602 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.153557    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/579247bb-977d-4c89-99cd-2831cb2ed824-archives-path\") pod \"periodic-gathering-zs5lf-ptq4h\" (UID: \"579247bb-977d-4c89-99cd-2831cb2ed824\") " pod="openshift-insights/periodic-gathering-zs5lf-ptq4h"
Oct 30 19:06:22.153602 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.153589    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9f98e875-33c4-473e-889b-223453fdd51b-image-registry-private-configuration\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.153677 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.153607    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9f98e875-33c4-473e-889b-223453fdd51b-registry-certificates\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.153677 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.153631    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/579247bb-977d-4c89-99cd-2831cb2ed824-service-ca-bundle\") pod \"periodic-gathering-zs5lf-ptq4h\" (UID: \"579247bb-977d-4c89-99cd-2831cb2ed824\") " pod="openshift-insights/periodic-gathering-zs5lf-ptq4h"
Oct 30 19:06:22.153798 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.153780    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9f98e875-33c4-473e-889b-223453fdd51b-bound-sa-token\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.153845 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.153815    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9f98e875-33c4-473e-889b-223453fdd51b-ca-trust-extracted\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.153925 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.153913    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6vsc\" (UniqueName: \"kubernetes.io/projected/5e25a61d-8a82-40ba-aad3-81ad9f762df7-kube-api-access-x6vsc\") pod \"downloads-6dd789c8b7-v6lqx\" (UID: \"5e25a61d-8a82-40ba-aad3-81ad9f762df7\") " pod="openshift-console/downloads-6dd789c8b7-v6lqx"
Oct 30 19:06:22.153978 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.153942    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9f98e875-33c4-473e-889b-223453fdd51b-installation-pull-secrets\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.153978 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.153928    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/579247bb-977d-4c89-99cd-2831cb2ed824-archives-path\") pod \"periodic-gathering-zs5lf-ptq4h\" (UID: \"579247bb-977d-4c89-99cd-2831cb2ed824\") " pod="openshift-insights/periodic-gathering-zs5lf-ptq4h"
Oct 30 19:06:22.154041 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.154017    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpr29\" (UniqueName: \"kubernetes.io/projected/579247bb-977d-4c89-99cd-2831cb2ed824-kube-api-access-gpr29\") pod \"periodic-gathering-zs5lf-ptq4h\" (UID: \"579247bb-977d-4c89-99cd-2831cb2ed824\") " pod="openshift-insights/periodic-gathering-zs5lf-ptq4h"
Oct 30 19:06:22.154074 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.154058    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9f98e875-33c4-473e-889b-223453fdd51b-registry-tls\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.154125 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.154106    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/579247bb-977d-4c89-99cd-2831cb2ed824-trusted-ca-bundle\") pod \"periodic-gathering-zs5lf-ptq4h\" (UID: \"579247bb-977d-4c89-99cd-2831cb2ed824\") " pod="openshift-insights/periodic-gathering-zs5lf-ptq4h"
Oct 30 19:06:22.154226 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.154210    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/579247bb-977d-4c89-99cd-2831cb2ed824-service-ca-bundle\") pod \"periodic-gathering-zs5lf-ptq4h\" (UID: \"579247bb-977d-4c89-99cd-2831cb2ed824\") " pod="openshift-insights/periodic-gathering-zs5lf-ptq4h"
Oct 30 19:06:22.154983 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.154965    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/579247bb-977d-4c89-99cd-2831cb2ed824-trusted-ca-bundle\") pod \"periodic-gathering-zs5lf-ptq4h\" (UID: \"579247bb-977d-4c89-99cd-2831cb2ed824\") " pod="openshift-insights/periodic-gathering-zs5lf-ptq4h"
Oct 30 19:06:22.173782 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.173748    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpr29\" (UniqueName: \"kubernetes.io/projected/579247bb-977d-4c89-99cd-2831cb2ed824-kube-api-access-gpr29\") pod \"periodic-gathering-zs5lf-ptq4h\" (UID: \"579247bb-977d-4c89-99cd-2831cb2ed824\") " pod="openshift-insights/periodic-gathering-zs5lf-ptq4h"
Oct 30 19:06:22.255246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.255142    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9f98e875-33c4-473e-889b-223453fdd51b-bound-sa-token\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.255246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.255203    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9f98e875-33c4-473e-889b-223453fdd51b-ca-trust-extracted\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.255246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.255236    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6vsc\" (UniqueName: \"kubernetes.io/projected/5e25a61d-8a82-40ba-aad3-81ad9f762df7-kube-api-access-x6vsc\") pod \"downloads-6dd789c8b7-v6lqx\" (UID: \"5e25a61d-8a82-40ba-aad3-81ad9f762df7\") " pod="openshift-console/downloads-6dd789c8b7-v6lqx"
Oct 30 19:06:22.255512 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.255259    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9f98e875-33c4-473e-889b-223453fdd51b-installation-pull-secrets\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.255512 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.255289    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9f98e875-33c4-473e-889b-223453fdd51b-registry-tls\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.255745 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.255710    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9f98e875-33c4-473e-889b-223453fdd51b-ca-trust-extracted\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.255853 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.255819    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxj9z\" (UniqueName: \"kubernetes.io/projected/9f98e875-33c4-473e-889b-223453fdd51b-kube-api-access-hxj9z\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.255853 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.255847    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8f4e6b19-4c02-43e5-ac9b-fb80af56e5f0-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-ztp7b\" (UID: \"8f4e6b19-4c02-43e5-ac9b-fb80af56e5f0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-ztp7b"
Oct 30 19:06:22.255929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.255873    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f98e875-33c4-473e-889b-223453fdd51b-trusted-ca\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.255929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.255915    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e25a61d-8a82-40ba-aad3-81ad9f762df7-tmp\") pod \"downloads-6dd789c8b7-v6lqx\" (UID: \"5e25a61d-8a82-40ba-aad3-81ad9f762df7\") " pod="openshift-console/downloads-6dd789c8b7-v6lqx"
Oct 30 19:06:22.256005 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.255954    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9f98e875-33c4-473e-889b-223453fdd51b-image-registry-private-configuration\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.256005 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.255976    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9f98e875-33c4-473e-889b-223453fdd51b-registry-certificates\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.256657 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.256630    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5e25a61d-8a82-40ba-aad3-81ad9f762df7-tmp\") pod \"downloads-6dd789c8b7-v6lqx\" (UID: \"5e25a61d-8a82-40ba-aad3-81ad9f762df7\") " pod="openshift-console/downloads-6dd789c8b7-v6lqx"
Oct 30 19:06:22.256822 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.256801    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9f98e875-33c4-473e-889b-223453fdd51b-registry-certificates\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.257198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.257115    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9f98e875-33c4-473e-889b-223453fdd51b-trusted-ca\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.258232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.258215    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9f98e875-33c4-473e-889b-223453fdd51b-installation-pull-secrets\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.258589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.258564    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9f98e875-33c4-473e-889b-223453fdd51b-registry-tls\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.259109 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.259085    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9f98e875-33c4-473e-889b-223453fdd51b-image-registry-private-configuration\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.259545 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.259520    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8f4e6b19-4c02-43e5-ac9b-fb80af56e5f0-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-ztp7b\" (UID: \"8f4e6b19-4c02-43e5-ac9b-fb80af56e5f0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-ztp7b"
Oct 30 19:06:22.271190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.271149    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9f98e875-33c4-473e-889b-223453fdd51b-bound-sa-token\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.272061 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.272040    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxj9z\" (UniqueName: \"kubernetes.io/projected/9f98e875-33c4-473e-889b-223453fdd51b-kube-api-access-hxj9z\") pod \"image-registry-6b4449ff4b-jmtvm\" (UID: \"9f98e875-33c4-473e-889b-223453fdd51b\") " pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.272142 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.272040    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6vsc\" (UniqueName: \"kubernetes.io/projected/5e25a61d-8a82-40ba-aad3-81ad9f762df7-kube-api-access-x6vsc\") pod \"downloads-6dd789c8b7-v6lqx\" (UID: \"5e25a61d-8a82-40ba-aad3-81ad9f762df7\") " pod="openshift-console/downloads-6dd789c8b7-v6lqx"
Oct 30 19:06:22.321702 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.321676    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-zs5lf-ptq4h"
Oct 30 19:06:22.339202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.339157    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-ztp7b"
Oct 30 19:06:22.348296 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.348274    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-v6lqx"
Oct 30 19:06:22.432202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.429763    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:22.563570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.563486    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ln5ql"
Oct 30 19:06:22.604197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.601526    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-zs5lf-ptq4h"]
Oct 30 19:06:22.610414 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:22.610388    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod579247bb_977d_4c89_99cd_2831cb2ed824.slice/crio-2203c9287dc1d528a5cb4a546fedb934fe37883a62ea1b90ed9bf4af3dd59476 WatchSource:0}: Error finding container 2203c9287dc1d528a5cb4a546fedb934fe37883a62ea1b90ed9bf4af3dd59476: Status 404 returned error can't find the container with id 2203c9287dc1d528a5cb4a546fedb934fe37883a62ea1b90ed9bf4af3dd59476
Oct 30 19:06:22.632660 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.632636    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-ztp7b"]
Oct 30 19:06:22.639743 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:22.639704    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f4e6b19_4c02_43e5_ac9b_fb80af56e5f0.slice/crio-63bd4d3ad08aa80fb4d1ab2be076018cbfecf334175f78df2a8d979296a61e7e WatchSource:0}: Error finding container 63bd4d3ad08aa80fb4d1ab2be076018cbfecf334175f78df2a8d979296a61e7e: Status 404 returned error can't find the container with id 63bd4d3ad08aa80fb4d1ab2be076018cbfecf334175f78df2a8d979296a61e7e
Oct 30 19:06:22.661441 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.661417    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-v6lqx"]
Oct 30 19:06:22.664710 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:22.664687    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e25a61d_8a82_40ba_aad3_81ad9f762df7.slice/crio-d3389dc47b3ddda1f064fb20bf8ec195bbfc8c1d765cf8086240bcf4e2f6a96e WatchSource:0}: Error finding container d3389dc47b3ddda1f064fb20bf8ec195bbfc8c1d765cf8086240bcf4e2f6a96e: Status 404 returned error can't find the container with id d3389dc47b3ddda1f064fb20bf8ec195bbfc8c1d765cf8086240bcf4e2f6a96e
Oct 30 19:06:22.715966 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:22.715890    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b4449ff4b-jmtvm"]
Oct 30 19:06:22.719580 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:22.719543    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f98e875_33c4_473e_889b_223453fdd51b.slice/crio-f22643a53778b0f7298cbce1558c64d39986ed6fe8431982bd962568430c19d9 WatchSource:0}: Error finding container f22643a53778b0f7298cbce1558c64d39986ed6fe8431982bd962568430c19d9: Status 404 returned error can't find the container with id f22643a53778b0f7298cbce1558c64d39986ed6fe8431982bd962568430c19d9
Oct 30 19:06:23.454303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:23.454272    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-v86zw"
Oct 30 19:06:23.485392 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:23.485298    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-zs5lf-ptq4h" event={"ID":"579247bb-977d-4c89-99cd-2831cb2ed824","Type":"ContainerStarted","Data":"d79d8f63c0d1ed1531a4822833d18b269cef9344b46d7bb73f108a14277fefd4"}
Oct 30 19:06:23.485392 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:23.485351    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-zs5lf-ptq4h" event={"ID":"579247bb-977d-4c89-99cd-2831cb2ed824","Type":"ContainerStarted","Data":"2203c9287dc1d528a5cb4a546fedb934fe37883a62ea1b90ed9bf4af3dd59476"}
Oct 30 19:06:23.493425 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:23.493388    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm" event={"ID":"9f98e875-33c4-473e-889b-223453fdd51b","Type":"ContainerStarted","Data":"f4a93db248d98dfd7791b9a4df3ec2da02a9158fa5218bbd1fe7cad65e8dea5f"}
Oct 30 19:06:23.493561 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:23.493489    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm" event={"ID":"9f98e875-33c4-473e-889b-223453fdd51b","Type":"ContainerStarted","Data":"f22643a53778b0f7298cbce1558c64d39986ed6fe8431982bd962568430c19d9"}
Oct 30 19:06:23.494140 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:23.494109    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:23.499488 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:23.499463    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-v6lqx" event={"ID":"5e25a61d-8a82-40ba-aad3-81ad9f762df7","Type":"ContainerStarted","Data":"d3389dc47b3ddda1f064fb20bf8ec195bbfc8c1d765cf8086240bcf4e2f6a96e"}
Oct 30 19:06:23.500846 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:23.500823    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-ztp7b" event={"ID":"8f4e6b19-4c02-43e5-ac9b-fb80af56e5f0","Type":"ContainerStarted","Data":"63bd4d3ad08aa80fb4d1ab2be076018cbfecf334175f78df2a8d979296a61e7e"}
Oct 30 19:06:23.505335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:23.504983    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-zs5lf-ptq4h" podStartSLOduration=2.504968364 podStartE2EDuration="2.504968364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:23.504528041 +0000 UTC m=+15.245519133" watchObservedRunningTime="2026-10-30 19:06:23.504968364 +0000 UTC m=+15.245959444"
Oct 30 19:06:23.527289 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:23.527238    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm" podStartSLOduration=1.5272207020000002 podStartE2EDuration="1.527220702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:23.526306201 +0000 UTC m=+15.267297293" watchObservedRunningTime="2026-10-30 19:06:23.527220702 +0000 UTC m=+15.268211783"
Oct 30 19:06:23.834103 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:23.834081    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v86zw_ce07e9b3-5e98-4d16-8158-afba6fe42281/dns/0.log"
Oct 30 19:06:24.033928 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:24.033898    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v86zw_ce07e9b3-5e98-4d16-8158-afba6fe42281/kube-rbac-proxy/0.log"
Oct 30 19:06:24.433682 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:24.433649    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8jrs8_94709da2-704a-421c-aa2f-1844db5bec96/dns-node-resolver/0.log"
Oct 30 19:06:24.507219 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:24.507183    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-ztp7b" event={"ID":"8f4e6b19-4c02-43e5-ac9b-fb80af56e5f0","Type":"ContainerStarted","Data":"0e645e9a988b8eb4ec99efe9e27b065cdcf7ab5b4e368afa4fa3827333e5b3bc"}
Oct 30 19:06:24.526523 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:24.526427    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-ztp7b" podStartSLOduration=2.349375325 podStartE2EDuration="3.526411573s" totalImagesPullingTime="1.177036248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:24.52568033 +0000 UTC m=+16.266671410" watchObservedRunningTime="2026-10-30 19:06:24.526411573 +0000 UTC m=+16.267402654"
Oct 30 19:06:24.833508 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:24.833478    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6b4449ff4b-jmtvm_9f98e875-33c4-473e-889b-223453fdd51b/registry/0.log"
Oct 30 19:06:25.433614 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:25.433580    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2z2fd_838f8f5e-ac73-4280-8ed3-173a51c42b9e/node-ca/0.log"
Oct 30 19:06:25.512349 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:25.511336    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-ztp7b"
Oct 30 19:06:25.516233 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:25.516207    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-ztp7b"
Oct 30 19:06:26.433269 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.433233    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xtj4p_7411c16b-533b-4152-bbe6-52262afa37da/serve-healthcheck-canary/0.log"
Oct 30 19:06:26.572134 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.572090    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-krs4x"]
Oct 30 19:06:26.574391 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.574371    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-krs4x"
Oct 30 19:06:26.577775 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.577659    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 19:06:26.577775 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.577716    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 19:06:26.577912 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.577884    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 19:06:26.577958 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.577912    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 19:06:26.577958 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.577916    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-55x4d\""
Oct 30 19:06:26.578032 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.577965    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 19:06:26.584779 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.584758    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-krs4x"]
Oct 30 19:06:26.701367 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.701278    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef933f88-83c6-4616-80de-e30974037645-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-krs4x\" (UID: \"ef933f88-83c6-4616-80de-e30974037645\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-krs4x"
Oct 30 19:06:26.701367 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.701350    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg65t\" (UniqueName: \"kubernetes.io/projected/ef933f88-83c6-4616-80de-e30974037645-kube-api-access-qg65t\") pod \"prometheus-operator-9cd89b74c-krs4x\" (UID: \"ef933f88-83c6-4616-80de-e30974037645\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-krs4x"
Oct 30 19:06:26.701556 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.701463    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef933f88-83c6-4616-80de-e30974037645-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-krs4x\" (UID: \"ef933f88-83c6-4616-80de-e30974037645\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-krs4x"
Oct 30 19:06:26.701556 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.701505    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef933f88-83c6-4616-80de-e30974037645-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-krs4x\" (UID: \"ef933f88-83c6-4616-80de-e30974037645\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-krs4x"
Oct 30 19:06:26.802319 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.802286    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef933f88-83c6-4616-80de-e30974037645-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-krs4x\" (UID: \"ef933f88-83c6-4616-80de-e30974037645\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-krs4x"
Oct 30 19:06:26.802506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.802354    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg65t\" (UniqueName: \"kubernetes.io/projected/ef933f88-83c6-4616-80de-e30974037645-kube-api-access-qg65t\") pod \"prometheus-operator-9cd89b74c-krs4x\" (UID: \"ef933f88-83c6-4616-80de-e30974037645\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-krs4x"
Oct 30 19:06:26.802506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.802408    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef933f88-83c6-4616-80de-e30974037645-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-krs4x\" (UID: \"ef933f88-83c6-4616-80de-e30974037645\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-krs4x"
Oct 30 19:06:26.802506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.802441    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef933f88-83c6-4616-80de-e30974037645-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-krs4x\" (UID: \"ef933f88-83c6-4616-80de-e30974037645\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-krs4x"
Oct 30 19:06:26.803062 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.803041    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef933f88-83c6-4616-80de-e30974037645-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-krs4x\" (UID: \"ef933f88-83c6-4616-80de-e30974037645\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-krs4x"
Oct 30 19:06:26.805803 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.805778    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/ef933f88-83c6-4616-80de-e30974037645-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-krs4x\" (UID: \"ef933f88-83c6-4616-80de-e30974037645\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-krs4x"
Oct 30 19:06:26.805912 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.805829    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef933f88-83c6-4616-80de-e30974037645-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-krs4x\" (UID: \"ef933f88-83c6-4616-80de-e30974037645\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-krs4x"
Oct 30 19:06:26.814507 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.814484    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg65t\" (UniqueName: \"kubernetes.io/projected/ef933f88-83c6-4616-80de-e30974037645-kube-api-access-qg65t\") pod \"prometheus-operator-9cd89b74c-krs4x\" (UID: \"ef933f88-83c6-4616-80de-e30974037645\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-krs4x"
Oct 30 19:06:26.889878 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:26.889849    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-krs4x"
Oct 30 19:06:27.168068 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:27.168034    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-krs4x"]
Oct 30 19:06:27.171455 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:27.171424    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef933f88_83c6_4616_80de_e30974037645.slice/crio-8833abaf96372963696329d53207ecee928b8c8d5c19cdc01d2763a5d50d3a40 WatchSource:0}: Error finding container 8833abaf96372963696329d53207ecee928b8c8d5c19cdc01d2763a5d50d3a40: Status 404 returned error can't find the container with id 8833abaf96372963696329d53207ecee928b8c8d5c19cdc01d2763a5d50d3a40
Oct 30 19:06:27.523473 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:27.523396    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-krs4x" event={"ID":"ef933f88-83c6-4616-80de-e30974037645","Type":"ContainerStarted","Data":"8833abaf96372963696329d53207ecee928b8c8d5c19cdc01d2763a5d50d3a40"}
Oct 30 19:06:28.532184 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:28.532138    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-krs4x" event={"ID":"ef933f88-83c6-4616-80de-e30974037645","Type":"ContainerStarted","Data":"68b933e530c36ba5ec1b1169ef95205dca751f24849d3f26143911eda4bee822"}
Oct 30 19:06:29.537907 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:29.537862    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-krs4x" event={"ID":"ef933f88-83c6-4616-80de-e30974037645","Type":"ContainerStarted","Data":"daca819799a250ca1c0e55487aa7c193a5a562da192ca814a810f3fbd9a1f2eb"}
Oct 30 19:06:32.130459 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.130399    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-krs4x" podStartSLOduration=4.849737865 podStartE2EDuration="6.130378547s" totalImagesPullingTime="1.280640682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:29.555529176 +0000 UTC m=+21.296520272" watchObservedRunningTime="2026-10-30 19:06:32.130378547 +0000 UTC m=+23.871369629"
Oct 30 19:06:32.131190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.131149    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-88zd5"]
Oct 30 19:06:32.134164 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.134137    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.165252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.165209    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 19:06:32.165252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.165229    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-n985b\""
Oct 30 19:06:32.165401 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.165258    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 19:06:32.205148 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.205116    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 19:06:32.248106 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.248073    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/caf4897d-8696-4083-b8d5-77714c6162e9-node-exporter-tls\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.248252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.248136    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/caf4897d-8696-4083-b8d5-77714c6162e9-node-exporter-accelerators-collector-config\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.248252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.248165    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/caf4897d-8696-4083-b8d5-77714c6162e9-sys\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.248252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.248209    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lvp5\" (UniqueName: \"kubernetes.io/projected/caf4897d-8696-4083-b8d5-77714c6162e9-kube-api-access-9lvp5\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.248252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.248239    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/caf4897d-8696-4083-b8d5-77714c6162e9-node-exporter-textfile\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.248406 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.248262    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/caf4897d-8696-4083-b8d5-77714c6162e9-root\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.248406 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.248277    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/caf4897d-8696-4083-b8d5-77714c6162e9-node-exporter-wtmp\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.248406 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.248299    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/caf4897d-8696-4083-b8d5-77714c6162e9-metrics-client-ca\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.248406 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.248334    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/caf4897d-8696-4083-b8d5-77714c6162e9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.349645 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.349607    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/caf4897d-8696-4083-b8d5-77714c6162e9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.349813 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.349665    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/caf4897d-8696-4083-b8d5-77714c6162e9-node-exporter-tls\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.349813 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.349711    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/caf4897d-8696-4083-b8d5-77714c6162e9-node-exporter-accelerators-collector-config\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.349813 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.349742    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/caf4897d-8696-4083-b8d5-77714c6162e9-sys\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.349813 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.349770    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lvp5\" (UniqueName: \"kubernetes.io/projected/caf4897d-8696-4083-b8d5-77714c6162e9-kube-api-access-9lvp5\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.349813 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.349800    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/caf4897d-8696-4083-b8d5-77714c6162e9-node-exporter-textfile\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.350056 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.349830    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/caf4897d-8696-4083-b8d5-77714c6162e9-root\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.350056 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.349873    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/caf4897d-8696-4083-b8d5-77714c6162e9-sys\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.350056 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.349889    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/caf4897d-8696-4083-b8d5-77714c6162e9-root\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.350056 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.349915    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/caf4897d-8696-4083-b8d5-77714c6162e9-node-exporter-wtmp\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.350056 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.349940    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/caf4897d-8696-4083-b8d5-77714c6162e9-metrics-client-ca\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.350289 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.350156    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/caf4897d-8696-4083-b8d5-77714c6162e9-node-exporter-wtmp\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.350883 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.350603    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/caf4897d-8696-4083-b8d5-77714c6162e9-node-exporter-accelerators-collector-config\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.350883 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.350841    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/caf4897d-8696-4083-b8d5-77714c6162e9-node-exporter-textfile\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.351198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.351155    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/caf4897d-8696-4083-b8d5-77714c6162e9-metrics-client-ca\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.352286 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.352267    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/caf4897d-8696-4083-b8d5-77714c6162e9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.352453 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.352436    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/caf4897d-8696-4083-b8d5-77714c6162e9-node-exporter-tls\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.362354 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.362332    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lvp5\" (UniqueName: \"kubernetes.io/projected/caf4897d-8696-4083-b8d5-77714c6162e9-kube-api-access-9lvp5\") pod \"node-exporter-88zd5\" (UID: \"caf4897d-8696-4083-b8d5-77714c6162e9\") " pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.449102 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.449027    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-88zd5"
Oct 30 19:06:32.461289 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:06:32.461263    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaf4897d_8696_4083_b8d5_77714c6162e9.slice/crio-431ddf5e91c97fb026e9c7f64f1c1cdabe812d2fbbe3e89838d77567bf79450d WatchSource:0}: Error finding container 431ddf5e91c97fb026e9c7f64f1c1cdabe812d2fbbe3e89838d77567bf79450d: Status 404 returned error can't find the container with id 431ddf5e91c97fb026e9c7f64f1c1cdabe812d2fbbe3e89838d77567bf79450d
Oct 30 19:06:32.550265 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:32.550227    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-88zd5" event={"ID":"caf4897d-8696-4083-b8d5-77714c6162e9","Type":"ContainerStarted","Data":"431ddf5e91c97fb026e9c7f64f1c1cdabe812d2fbbe3e89838d77567bf79450d"}
Oct 30 19:06:33.561084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:33.561051    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-88zd5" event={"ID":"caf4897d-8696-4083-b8d5-77714c6162e9","Type":"ContainerStarted","Data":"0b6b9a148742f7daab6796df6a9152112d109898e1d36fe36d5cd204f07654ca"}
Oct 30 19:06:34.566757 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:34.566717    4524 generic.go:356] "Generic (PLEG): container finished" podID="caf4897d-8696-4083-b8d5-77714c6162e9" containerID="0b6b9a148742f7daab6796df6a9152112d109898e1d36fe36d5cd204f07654ca" exitCode=0
Oct 30 19:06:34.567259 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:34.566786    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-88zd5" event={"ID":"caf4897d-8696-4083-b8d5-77714c6162e9","Type":"ContainerDied","Data":"0b6b9a148742f7daab6796df6a9152112d109898e1d36fe36d5cd204f07654ca"}
Oct 30 19:06:35.573213 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:35.573163    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-88zd5" event={"ID":"caf4897d-8696-4083-b8d5-77714c6162e9","Type":"ContainerStarted","Data":"1d10554c4abbadfd4d6a1fe3ea849684d4e4364709e0cc07d7a4831e28d6410c"}
Oct 30 19:06:35.573671 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:35.573232    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-88zd5" event={"ID":"caf4897d-8696-4083-b8d5-77714c6162e9","Type":"ContainerStarted","Data":"8dc28f8c0d3ac5b44443ff509ce252b8904997242c85ca78027ec0ee59576d98"}
Oct 30 19:06:37.957020 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:37.956987    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z2dbh"
Oct 30 19:06:37.974911 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:37.974860    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-88zd5" podStartSLOduration=4.913758544 podStartE2EDuration="5.974846674s" totalImagesPullingTime="980.109694ms" totalInitContainerRuntime="80.978436ms" isStatefulPod=true podCreationTimestamp="2026-10-30 19:06:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:35.595256421 +0000 UTC m=+27.336247511" watchObservedRunningTime="2026-10-30 19:06:37.974846674 +0000 UTC m=+29.715837766"
Oct 30 19:06:39.589709 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:39.589676    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-v6lqx" event={"ID":"5e25a61d-8a82-40ba-aad3-81ad9f762df7","Type":"ContainerStarted","Data":"f8977ac3cdec3ee627a4f6deb40d98f53efe24cdc6e6c82e20fac892933fa4f5"}
Oct 30 19:06:39.590114 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:39.590084    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-v6lqx"
Oct 30 19:06:39.599330 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:39.599264    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-v6lqx"
Oct 30 19:06:39.608612 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:39.608565    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-v6lqx" podStartSLOduration=1.138721889 podStartE2EDuration="17.608550894s" totalImagesPullingTime="16.469829005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:39.60853691 +0000 UTC m=+31.349527990" watchObservedRunningTime="2026-10-30 19:06:39.608550894 +0000 UTC m=+31.349541974"
Oct 30 19:06:45.516699 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:45.516663    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6b4449ff4b-jmtvm"
Oct 30 19:06:45.638501 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:06:45.638410    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:06:54.176468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.176287    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-75z8h-spdtl"]
Oct 30 19:06:54.177060 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.177008    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-75z8h-spdtl" podUID="2ac4e9e2-d815-4b3b-86cd-db537f984e6c" containerName="insights-gathering" containerID="cri-o://b44637b0d0eef1f1754b5656466bbc08de9d3d662fadcf883d9d596602a91542" gracePeriod=30
Oct 30 19:06:54.477031 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.476998    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:06:54.544071 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.544041    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-service-ca-bundle\") pod \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") "
Oct 30 19:06:54.544213 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.544084    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-kube-api-access-wqgn5\" (UniqueName: \"kubernetes.io/projected/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-kube-api-access-wqgn5\") pod \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") "
Oct 30 19:06:54.544213 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.544127    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-trusted-ca-bundle\") pod \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") "
Oct 30 19:06:54.544213 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.544155    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-archives-path\") pod \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\" (UID: \"2ac4e9e2-d815-4b3b-86cd-db537f984e6c\") "
Oct 30 19:06:54.544540 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.544517    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-trusted-ca-bundle" pod "2ac4e9e2-d815-4b3b-86cd-db537f984e6c" (UID: "2ac4e9e2-d815-4b3b-86cd-db537f984e6c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:54.544598 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.544515    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-service-ca-bundle" pod "2ac4e9e2-d815-4b3b-86cd-db537f984e6c" (UID: "2ac4e9e2-d815-4b3b-86cd-db537f984e6c"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:06:54.544598 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.544560    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-archives-path" pod "2ac4e9e2-d815-4b3b-86cd-db537f984e6c" (UID: "2ac4e9e2-d815-4b3b-86cd-db537f984e6c"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:06:54.546334 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.546312    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-kube-api-access-wqgn5" pod "2ac4e9e2-d815-4b3b-86cd-db537f984e6c" (UID: "2ac4e9e2-d815-4b3b-86cd-db537f984e6c"). InnerVolumeSpecName "kube-api-access-wqgn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:06:54.645093 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.645071    4524 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-service-ca-bundle\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:54.645190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.645094    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqgn5\" (UniqueName: \"kubernetes.io/projected/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-kube-api-access-wqgn5\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:54.645190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.645105    4524 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-trusted-ca-bundle\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:54.645190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.645114    4524 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/2ac4e9e2-d815-4b3b-86cd-db537f984e6c-archives-path\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:06:54.650863 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.650838    4524 generic.go:356] "Generic (PLEG): container finished" podID="2ac4e9e2-d815-4b3b-86cd-db537f984e6c" containerID="b44637b0d0eef1f1754b5656466bbc08de9d3d662fadcf883d9d596602a91542" exitCode=2
Oct 30 19:06:54.650942 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.650891    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-75z8h-spdtl"
Oct 30 19:06:54.650942 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.650906    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-75z8h-spdtl" event={"ID":"2ac4e9e2-d815-4b3b-86cd-db537f984e6c","Type":"ContainerDied","Data":"b44637b0d0eef1f1754b5656466bbc08de9d3d662fadcf883d9d596602a91542"}
Oct 30 19:06:54.651008 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.650952    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-75z8h-spdtl" event={"ID":"2ac4e9e2-d815-4b3b-86cd-db537f984e6c","Type":"ContainerDied","Data":"e00ec3b69fb49a9276c87d5dc5a8dd5226a2db07f48ab9f9d4781dcf0c682ccc"}
Oct 30 19:06:54.651008 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.650978    4524 scope.go:122] "RemoveContainer" containerID="b44637b0d0eef1f1754b5656466bbc08de9d3d662fadcf883d9d596602a91542"
Oct 30 19:06:54.658751 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.658732    4524 scope.go:122] "RemoveContainer" containerID="b44637b0d0eef1f1754b5656466bbc08de9d3d662fadcf883d9d596602a91542"
Oct 30 19:06:54.659009 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:06:54.658990    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b44637b0d0eef1f1754b5656466bbc08de9d3d662fadcf883d9d596602a91542\": container with ID starting with b44637b0d0eef1f1754b5656466bbc08de9d3d662fadcf883d9d596602a91542 not found: ID does not exist" containerID="b44637b0d0eef1f1754b5656466bbc08de9d3d662fadcf883d9d596602a91542"
Oct 30 19:06:54.659059 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.659018    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b44637b0d0eef1f1754b5656466bbc08de9d3d662fadcf883d9d596602a91542"} err="failed to get container status \"b44637b0d0eef1f1754b5656466bbc08de9d3d662fadcf883d9d596602a91542\": rpc error: code = NotFound desc = could not find container \"b44637b0d0eef1f1754b5656466bbc08de9d3d662fadcf883d9d596602a91542\": container with ID starting with b44637b0d0eef1f1754b5656466bbc08de9d3d662fadcf883d9d596602a91542 not found: ID does not exist"
Oct 30 19:06:54.670036 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.670014    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-75z8h-spdtl"]
Oct 30 19:06:54.680478 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:54.680454    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-75z8h-spdtl"]
Oct 30 19:06:56.387042 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:06:56.387005    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2ac4e9e2-d815-4b3b-86cd-db537f984e6c" path="/var/lib/kubelet/pods/2ac4e9e2-d815-4b3b-86cd-db537f984e6c/volumes"
Oct 30 19:07:15.623765 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:07:15.623718    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:07:16.518471 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.518435    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"]
Oct 30 19:07:16.518714 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.518702    4524 cpu_manager.go:414] "containerMap: removing container" podUID="2ac4e9e2-d815-4b3b-86cd-db537f984e6c" containerName="insights-gathering"
Oct 30 19:07:16.518762 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.518716    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ac4e9e2-d815-4b3b-86cd-db537f984e6c" containerName="insights-gathering"
Oct 30 19:07:16.518795 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.518771    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ac4e9e2-d815-4b3b-86cd-db537f984e6c" containerName="insights-gathering"
Oct 30 19:07:16.555238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.555202    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"]
Oct 30 19:07:16.555426 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.555288    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.577699 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.577673    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/10e0b1be-f568-4df2-a87a-68468d34835b-telemeter-client-tls\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.577864 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.577708    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/10e0b1be-f568-4df2-a87a-68468d34835b-federate-client-tls\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.577864 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.577724    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e0b1be-f568-4df2-a87a-68468d34835b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.577864 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.577846    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10e0b1be-f568-4df2-a87a-68468d34835b-metrics-client-ca\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.578015 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.577888    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld69q\" (UniqueName: \"kubernetes.io/projected/10e0b1be-f568-4df2-a87a-68468d34835b-kube-api-access-ld69q\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.578015 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.577982    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10e0b1be-f568-4df2-a87a-68468d34835b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.578119 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.578033    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e0b1be-f568-4df2-a87a-68468d34835b-serving-certs-ca-bundle\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.578119 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.578110    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/10e0b1be-f568-4df2-a87a-68468d34835b-secret-telemeter-client\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.620989 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.620967    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 19:07:16.620989 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.620973    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 19:07:16.621199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.620974    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-sf88g\""
Oct 30 19:07:16.621199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.620971    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 19:07:16.621199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.621041    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 19:07:16.630563 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.630543    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 19:07:16.632393 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.632376    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 19:07:16.678764 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.678736    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/10e0b1be-f568-4df2-a87a-68468d34835b-federate-client-tls\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.678764 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.678767    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e0b1be-f568-4df2-a87a-68468d34835b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.679005 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.678792    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10e0b1be-f568-4df2-a87a-68468d34835b-metrics-client-ca\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.679005 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.678823    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld69q\" (UniqueName: \"kubernetes.io/projected/10e0b1be-f568-4df2-a87a-68468d34835b-kube-api-access-ld69q\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.679005 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.678919    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10e0b1be-f568-4df2-a87a-68468d34835b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.679005 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.678987    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e0b1be-f568-4df2-a87a-68468d34835b-serving-certs-ca-bundle\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.679754 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.679724    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/10e0b1be-f568-4df2-a87a-68468d34835b-metrics-client-ca\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.679902 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.679790    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e0b1be-f568-4df2-a87a-68468d34835b-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.679902 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.679800    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/10e0b1be-f568-4df2-a87a-68468d34835b-serving-certs-ca-bundle\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.679902 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.679853    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/10e0b1be-f568-4df2-a87a-68468d34835b-secret-telemeter-client\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.680073 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.679914    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/10e0b1be-f568-4df2-a87a-68468d34835b-telemeter-client-tls\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.681557 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.681529    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/10e0b1be-f568-4df2-a87a-68468d34835b-federate-client-tls\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.681658 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.681610    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/10e0b1be-f568-4df2-a87a-68468d34835b-secret-telemeter-client\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.681658 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.681615    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/10e0b1be-f568-4df2-a87a-68468d34835b-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.684411 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.684391    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/10e0b1be-f568-4df2-a87a-68468d34835b-telemeter-client-tls\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.690314 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.690297    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld69q\" (UniqueName: \"kubernetes.io/projected/10e0b1be-f568-4df2-a87a-68468d34835b-kube-api-access-ld69q\") pod \"telemeter-client-5bbccbcb8-p4h88\" (UID: \"10e0b1be-f568-4df2-a87a-68468d34835b\") " pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:16.904080 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:16.904048    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"
Oct 30 19:07:17.217510 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:17.217479    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5bbccbcb8-p4h88"]
Oct 30 19:07:17.220056 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:07:17.220029    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10e0b1be_f568_4df2_a87a_68468d34835b.slice/crio-22e0dd6bff12d13d0a596f6412807c1d71cc96bc96ffc06ef29f3f2ff533f950 WatchSource:0}: Error finding container 22e0dd6bff12d13d0a596f6412807c1d71cc96bc96ffc06ef29f3f2ff533f950: Status 404 returned error can't find the container with id 22e0dd6bff12d13d0a596f6412807c1d71cc96bc96ffc06ef29f3f2ff533f950
Oct 30 19:07:17.722979 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:17.722943    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88" event={"ID":"10e0b1be-f568-4df2-a87a-68468d34835b","Type":"ContainerStarted","Data":"22e0dd6bff12d13d0a596f6412807c1d71cc96bc96ffc06ef29f3f2ff533f950"}
Oct 30 19:07:19.733524 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:19.733489    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88" event={"ID":"10e0b1be-f568-4df2-a87a-68468d34835b","Type":"ContainerStarted","Data":"0e00b4481611507fcf5af7bde69fb191a53c78e2012ed087f865ce3a6491e2a6"}
Oct 30 19:07:21.741808 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:21.741767    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88" event={"ID":"10e0b1be-f568-4df2-a87a-68468d34835b","Type":"ContainerStarted","Data":"3e957e1aaa915654668e7241ab1cb2d023b0698da893b0ad2dd83b3bb7a47ccc"}
Oct 30 19:07:21.742215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:21.741838    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88" event={"ID":"10e0b1be-f568-4df2-a87a-68468d34835b","Type":"ContainerStarted","Data":"844202bec899846513b00e64ecc9b54a2a7e04bb3620b3b5ac954f5058a1c779"}
Oct 30 19:07:21.768933 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:21.768885    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5bbccbcb8-p4h88" podStartSLOduration=2.4005180360000002 podStartE2EDuration="5.768872252s" totalImagesPullingTime="3.368354216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:21.767967765 +0000 UTC m=+73.508958846" watchObservedRunningTime="2026-10-30 19:07:21.768872252 +0000 UTC m=+73.509863362"
Oct 30 19:07:45.623053 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:07:45.623013    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:07:47.824711 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:47.824683    4524 generic.go:356] "Generic (PLEG): container finished" podID="579247bb-977d-4c89-99cd-2831cb2ed824" containerID="d79d8f63c0d1ed1531a4822833d18b269cef9344b46d7bb73f108a14277fefd4" exitCode=0
Oct 30 19:07:47.824997 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:47.824759    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-zs5lf-ptq4h" event={"ID":"579247bb-977d-4c89-99cd-2831cb2ed824","Type":"ContainerDied","Data":"d79d8f63c0d1ed1531a4822833d18b269cef9344b46d7bb73f108a14277fefd4"}
Oct 30 19:07:48.949273 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:48.949248    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-zs5lf-ptq4h"
Oct 30 19:07:49.008330 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:49.008302    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/579247bb-977d-4c89-99cd-2831cb2ed824-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/579247bb-977d-4c89-99cd-2831cb2ed824-service-ca-bundle\") pod \"579247bb-977d-4c89-99cd-2831cb2ed824\" (UID: \"579247bb-977d-4c89-99cd-2831cb2ed824\") "
Oct 30 19:07:49.008454 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:49.008356    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/579247bb-977d-4c89-99cd-2831cb2ed824-kube-api-access-gpr29\" (UniqueName: \"kubernetes.io/projected/579247bb-977d-4c89-99cd-2831cb2ed824-kube-api-access-gpr29\") pod \"579247bb-977d-4c89-99cd-2831cb2ed824\" (UID: \"579247bb-977d-4c89-99cd-2831cb2ed824\") "
Oct 30 19:07:49.008454 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:49.008422    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/579247bb-977d-4c89-99cd-2831cb2ed824-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/579247bb-977d-4c89-99cd-2831cb2ed824-archives-path\") pod \"579247bb-977d-4c89-99cd-2831cb2ed824\" (UID: \"579247bb-977d-4c89-99cd-2831cb2ed824\") "
Oct 30 19:07:49.008572 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:49.008462    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/579247bb-977d-4c89-99cd-2831cb2ed824-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/579247bb-977d-4c89-99cd-2831cb2ed824-trusted-ca-bundle\") pod \"579247bb-977d-4c89-99cd-2831cb2ed824\" (UID: \"579247bb-977d-4c89-99cd-2831cb2ed824\") "
Oct 30 19:07:49.008677 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:49.008652    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/579247bb-977d-4c89-99cd-2831cb2ed824-service-ca-bundle" pod "579247bb-977d-4c89-99cd-2831cb2ed824" (UID: "579247bb-977d-4c89-99cd-2831cb2ed824"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:07:49.008787 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:49.008768    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/579247bb-977d-4c89-99cd-2831cb2ed824-archives-path" pod "579247bb-977d-4c89-99cd-2831cb2ed824" (UID: "579247bb-977d-4c89-99cd-2831cb2ed824"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:07:49.008870 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:49.008853    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/579247bb-977d-4c89-99cd-2831cb2ed824-trusted-ca-bundle" pod "579247bb-977d-4c89-99cd-2831cb2ed824" (UID: "579247bb-977d-4c89-99cd-2831cb2ed824"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:07:49.010351 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:49.010334    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/579247bb-977d-4c89-99cd-2831cb2ed824-kube-api-access-gpr29" pod "579247bb-977d-4c89-99cd-2831cb2ed824" (UID: "579247bb-977d-4c89-99cd-2831cb2ed824"). InnerVolumeSpecName "kube-api-access-gpr29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:07:49.109712 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:49.109692    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpr29\" (UniqueName: \"kubernetes.io/projected/579247bb-977d-4c89-99cd-2831cb2ed824-kube-api-access-gpr29\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:49.109712 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:49.109712    4524 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/579247bb-977d-4c89-99cd-2831cb2ed824-archives-path\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:49.109836 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:49.109722    4524 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/579247bb-977d-4c89-99cd-2831cb2ed824-trusted-ca-bundle\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:49.109836 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:49.109730    4524 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/579247bb-977d-4c89-99cd-2831cb2ed824-service-ca-bundle\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:07:49.830953 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:49.830919    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-zs5lf-ptq4h" event={"ID":"579247bb-977d-4c89-99cd-2831cb2ed824","Type":"ContainerDied","Data":"2203c9287dc1d528a5cb4a546fedb934fe37883a62ea1b90ed9bf4af3dd59476"}
Oct 30 19:07:49.831110 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:49.830965    4524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2203c9287dc1d528a5cb4a546fedb934fe37883a62ea1b90ed9bf4af3dd59476"
Oct 30 19:07:49.831110 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:07:49.830963    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-zs5lf-ptq4h"
Oct 30 19:08:15.623781 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:08:15.623736    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:08:45.622939 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:08:45.622906    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:09:15.626338 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:09:15.624482    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:09:45.622965 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:09:45.622930    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:10:15.622842 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:10:15.622802    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:10:21.881403 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:21.881369    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw"]
Oct 30 19:10:21.881891 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:21.881637    4524 cpu_manager.go:414] "containerMap: removing container" podUID="579247bb-977d-4c89-99cd-2831cb2ed824" containerName="insights-gathering"
Oct 30 19:10:21.881891 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:21.881647    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="579247bb-977d-4c89-99cd-2831cb2ed824" containerName="insights-gathering"
Oct 30 19:10:21.881891 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:21.881707    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="579247bb-977d-4c89-99cd-2831cb2ed824" containerName="insights-gathering"
Oct 30 19:10:21.884653 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:21.884637    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw"
Oct 30 19:10:21.888145 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:21.888123    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 19:10:21.888260 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:21.888161    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 19:10:21.888260 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:21.888219    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-5nwnm\""
Oct 30 19:10:21.894664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:21.894641    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw"]
Oct 30 19:10:22.007959 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:22.007936    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26a52b8d-a9d5-4e07-94fe-bd763370687c-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw\" (UID: \"26a52b8d-a9d5-4e07-94fe-bd763370687c\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw"
Oct 30 19:10:22.008048 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:22.007970    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frdbh\" (UniqueName: \"kubernetes.io/projected/26a52b8d-a9d5-4e07-94fe-bd763370687c-kube-api-access-frdbh\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw\" (UID: \"26a52b8d-a9d5-4e07-94fe-bd763370687c\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw"
Oct 30 19:10:22.008048 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:22.008041    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26a52b8d-a9d5-4e07-94fe-bd763370687c-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw\" (UID: \"26a52b8d-a9d5-4e07-94fe-bd763370687c\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw"
Oct 30 19:10:22.109164 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:22.109139    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26a52b8d-a9d5-4e07-94fe-bd763370687c-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw\" (UID: \"26a52b8d-a9d5-4e07-94fe-bd763370687c\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw"
Oct 30 19:10:22.109266 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:22.109211    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26a52b8d-a9d5-4e07-94fe-bd763370687c-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw\" (UID: \"26a52b8d-a9d5-4e07-94fe-bd763370687c\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw"
Oct 30 19:10:22.109266 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:22.109238    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frdbh\" (UniqueName: \"kubernetes.io/projected/26a52b8d-a9d5-4e07-94fe-bd763370687c-kube-api-access-frdbh\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw\" (UID: \"26a52b8d-a9d5-4e07-94fe-bd763370687c\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw"
Oct 30 19:10:22.109538 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:22.109520    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26a52b8d-a9d5-4e07-94fe-bd763370687c-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw\" (UID: \"26a52b8d-a9d5-4e07-94fe-bd763370687c\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw"
Oct 30 19:10:22.109608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:22.109594    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26a52b8d-a9d5-4e07-94fe-bd763370687c-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw\" (UID: \"26a52b8d-a9d5-4e07-94fe-bd763370687c\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw"
Oct 30 19:10:22.121046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:22.121028    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frdbh\" (UniqueName: \"kubernetes.io/projected/26a52b8d-a9d5-4e07-94fe-bd763370687c-kube-api-access-frdbh\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw\" (UID: \"26a52b8d-a9d5-4e07-94fe-bd763370687c\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw"
Oct 30 19:10:22.201047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:22.200986    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw"
Oct 30 19:10:22.471582 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:22.471519    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw"]
Oct 30 19:10:22.475032 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:10:22.475006    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26a52b8d_a9d5_4e07_94fe_bd763370687c.slice/crio-1e10408d2d1c4c9b9e94b1ee99e28c6785e81578b1cfad4ea93e13604571ce6a WatchSource:0}: Error finding container 1e10408d2d1c4c9b9e94b1ee99e28c6785e81578b1cfad4ea93e13604571ce6a: Status 404 returned error can't find the container with id 1e10408d2d1c4c9b9e94b1ee99e28c6785e81578b1cfad4ea93e13604571ce6a
Oct 30 19:10:23.320727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:23.320684    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw" event={"ID":"26a52b8d-a9d5-4e07-94fe-bd763370687c","Type":"ContainerStarted","Data":"1e10408d2d1c4c9b9e94b1ee99e28c6785e81578b1cfad4ea93e13604571ce6a"}
Oct 30 19:10:28.343349 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:28.343317    4524 generic.go:356] "Generic (PLEG): container finished" podID="26a52b8d-a9d5-4e07-94fe-bd763370687c" containerID="2716fc7fcb654dbe47c94e497d1aaa1291db91034a98ec45a02d55693c3f3ded" exitCode=0
Oct 30 19:10:28.343787 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:28.343345    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw" event={"ID":"26a52b8d-a9d5-4e07-94fe-bd763370687c","Type":"ContainerDied","Data":"2716fc7fcb654dbe47c94e497d1aaa1291db91034a98ec45a02d55693c3f3ded"}
Oct 30 19:10:31.356533 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:31.356490    4524 generic.go:356] "Generic (PLEG): container finished" podID="26a52b8d-a9d5-4e07-94fe-bd763370687c" containerID="5a4c92dfc933a9210f5445c35cbdbb6fa9cc3af595d764e278a2712a78a4f729" exitCode=0
Oct 30 19:10:31.356921 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:31.356555    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw" event={"ID":"26a52b8d-a9d5-4e07-94fe-bd763370687c","Type":"ContainerDied","Data":"5a4c92dfc933a9210f5445c35cbdbb6fa9cc3af595d764e278a2712a78a4f729"}
Oct 30 19:10:37.381881 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:37.381848    4524 generic.go:356] "Generic (PLEG): container finished" podID="26a52b8d-a9d5-4e07-94fe-bd763370687c" containerID="efc1899777a6336d567b95608b390e0453eede43b2b6f32c0154e1262a190038" exitCode=0
Oct 30 19:10:37.382291 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:37.381918    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw" event={"ID":"26a52b8d-a9d5-4e07-94fe-bd763370687c","Type":"ContainerDied","Data":"efc1899777a6336d567b95608b390e0453eede43b2b6f32c0154e1262a190038"}
Oct 30 19:10:38.510582 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:38.510559    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw"
Oct 30 19:10:38.639108 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:38.639054    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/26a52b8d-a9d5-4e07-94fe-bd763370687c-kube-api-access-frdbh\" (UniqueName: \"kubernetes.io/projected/26a52b8d-a9d5-4e07-94fe-bd763370687c-kube-api-access-frdbh\") pod \"26a52b8d-a9d5-4e07-94fe-bd763370687c\" (UID: \"26a52b8d-a9d5-4e07-94fe-bd763370687c\") "
Oct 30 19:10:38.639108 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:38.639091    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26a52b8d-a9d5-4e07-94fe-bd763370687c-bundle\" (UniqueName: \"kubernetes.io/empty-dir/26a52b8d-a9d5-4e07-94fe-bd763370687c-bundle\") pod \"26a52b8d-a9d5-4e07-94fe-bd763370687c\" (UID: \"26a52b8d-a9d5-4e07-94fe-bd763370687c\") "
Oct 30 19:10:38.639317 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:38.639114    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/26a52b8d-a9d5-4e07-94fe-bd763370687c-util\" (UniqueName: \"kubernetes.io/empty-dir/26a52b8d-a9d5-4e07-94fe-bd763370687c-util\") pod \"26a52b8d-a9d5-4e07-94fe-bd763370687c\" (UID: \"26a52b8d-a9d5-4e07-94fe-bd763370687c\") "
Oct 30 19:10:38.639710 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:38.639686    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26a52b8d-a9d5-4e07-94fe-bd763370687c-bundle" pod "26a52b8d-a9d5-4e07-94fe-bd763370687c" (UID: "26a52b8d-a9d5-4e07-94fe-bd763370687c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:10:38.641123 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:38.641099    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26a52b8d-a9d5-4e07-94fe-bd763370687c-kube-api-access-frdbh" pod "26a52b8d-a9d5-4e07-94fe-bd763370687c" (UID: "26a52b8d-a9d5-4e07-94fe-bd763370687c"). InnerVolumeSpecName "kube-api-access-frdbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:10:38.643244 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:38.643222    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26a52b8d-a9d5-4e07-94fe-bd763370687c-util" pod "26a52b8d-a9d5-4e07-94fe-bd763370687c" (UID: "26a52b8d-a9d5-4e07-94fe-bd763370687c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:10:38.740312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:38.740277    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frdbh\" (UniqueName: \"kubernetes.io/projected/26a52b8d-a9d5-4e07-94fe-bd763370687c-kube-api-access-frdbh\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:38.740312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:38.740311    4524 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/26a52b8d-a9d5-4e07-94fe-bd763370687c-bundle\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:38.740312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:38.740321    4524 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/26a52b8d-a9d5-4e07-94fe-bd763370687c-util\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:39.390082 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:39.390054    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw"
Oct 30 19:10:39.390285 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:39.390077    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651ctpklw" event={"ID":"26a52b8d-a9d5-4e07-94fe-bd763370687c","Type":"ContainerDied","Data":"1e10408d2d1c4c9b9e94b1ee99e28c6785e81578b1cfad4ea93e13604571ce6a"}
Oct 30 19:10:39.390285 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:39.390120    4524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e10408d2d1c4c9b9e94b1ee99e28c6785e81578b1cfad4ea93e13604571ce6a"
Oct 30 19:10:45.122359 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.122328    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-tmcx7"]
Oct 30 19:10:45.122766 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.122605    4524 cpu_manager.go:414] "containerMap: removing container" podUID="26a52b8d-a9d5-4e07-94fe-bd763370687c" containerName="util"
Oct 30 19:10:45.122766 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.122617    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26a52b8d-a9d5-4e07-94fe-bd763370687c" containerName="util"
Oct 30 19:10:45.122766 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.122631    4524 cpu_manager.go:414] "containerMap: removing container" podUID="26a52b8d-a9d5-4e07-94fe-bd763370687c" containerName="pull"
Oct 30 19:10:45.122766 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.122636    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26a52b8d-a9d5-4e07-94fe-bd763370687c" containerName="pull"
Oct 30 19:10:45.122766 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.122642    4524 cpu_manager.go:414] "containerMap: removing container" podUID="26a52b8d-a9d5-4e07-94fe-bd763370687c" containerName="extract"
Oct 30 19:10:45.122766 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.122648    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="26a52b8d-a9d5-4e07-94fe-bd763370687c" containerName="extract"
Oct 30 19:10:45.122766 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.122692    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="26a52b8d-a9d5-4e07-94fe-bd763370687c" containerName="extract"
Oct 30 19:10:45.167481 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.167453    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-tmcx7"]
Oct 30 19:10:45.167648 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.167585    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-tmcx7"
Oct 30 19:10:45.173222 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.173152    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Oct 30 19:10:45.173335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.173301    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:10:45.173572 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.173553    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-dngkx\""
Oct 30 19:10:45.182146 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.182126    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41615fe1-7a58-4121-aef8-854976b05bed-tmp\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-tmcx7\" (UID: \"41615fe1-7a58-4121-aef8-854976b05bed\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-tmcx7"
Oct 30 19:10:45.182252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.182180    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j27kh\" (UniqueName: \"kubernetes.io/projected/41615fe1-7a58-4121-aef8-854976b05bed-kube-api-access-j27kh\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-tmcx7\" (UID: \"41615fe1-7a58-4121-aef8-854976b05bed\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-tmcx7"
Oct 30 19:10:45.283257 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.283231    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41615fe1-7a58-4121-aef8-854976b05bed-tmp\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-tmcx7\" (UID: \"41615fe1-7a58-4121-aef8-854976b05bed\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-tmcx7"
Oct 30 19:10:45.283391 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.283279    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j27kh\" (UniqueName: \"kubernetes.io/projected/41615fe1-7a58-4121-aef8-854976b05bed-kube-api-access-j27kh\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-tmcx7\" (UID: \"41615fe1-7a58-4121-aef8-854976b05bed\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-tmcx7"
Oct 30 19:10:45.283572 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.283553    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41615fe1-7a58-4121-aef8-854976b05bed-tmp\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-tmcx7\" (UID: \"41615fe1-7a58-4121-aef8-854976b05bed\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-tmcx7"
Oct 30 19:10:45.297965 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.297941    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j27kh\" (UniqueName: \"kubernetes.io/projected/41615fe1-7a58-4121-aef8-854976b05bed-kube-api-access-j27kh\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-tmcx7\" (UID: \"41615fe1-7a58-4121-aef8-854976b05bed\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-tmcx7"
Oct 30 19:10:45.489811 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.489731    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-tmcx7"
Oct 30 19:10:45.623583 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:10:45.623540    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:10:45.787778 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:45.787673    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-tmcx7"]
Oct 30 19:10:45.791413 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:10:45.791386    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41615fe1_7a58_4121_aef8_854976b05bed.slice/crio-5e166d1948685dc879464d353194ec5de92a223d803423da9fd05bc66d45e805 WatchSource:0}: Error finding container 5e166d1948685dc879464d353194ec5de92a223d803423da9fd05bc66d45e805: Status 404 returned error can't find the container with id 5e166d1948685dc879464d353194ec5de92a223d803423da9fd05bc66d45e805
Oct 30 19:10:46.412906 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:46.412875    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-tmcx7" event={"ID":"41615fe1-7a58-4121-aef8-854976b05bed","Type":"ContainerStarted","Data":"5e166d1948685dc879464d353194ec5de92a223d803423da9fd05bc66d45e805"}
Oct 30 19:10:48.422043 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:48.422006    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-tmcx7" event={"ID":"41615fe1-7a58-4121-aef8-854976b05bed","Type":"ContainerStarted","Data":"03b442ca60e615393d045ae3644cef22ecc324ca4833866c290b73ae17decb05"}
Oct 30 19:10:48.443897 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:48.443850    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-tmcx7" podStartSLOduration=1.18365099 podStartE2EDuration="3.443836705s" totalImagesPullingTime="2.260185715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:10:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:10:48.442822003 +0000 UTC m=+280.183813092" watchObservedRunningTime="2026-10-30 19:10:48.443836705 +0000 UTC m=+280.184827781"
Oct 30 19:10:48.999800 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:48.999772    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-pvh7s"]
Oct 30 19:10:49.019698 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.019670    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-pvh7s"]
Oct 30 19:10:49.019850 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.019787    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-pvh7s"
Oct 30 19:10:49.027413 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.027393    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 19:10:49.030204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.030188    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 19:10:49.031627 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.031605    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-lh2bn\""
Oct 30 19:10:49.111415 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.111387    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlnqx\" (UniqueName: \"kubernetes.io/projected/31ccc096-b457-491c-a8cd-408865baaa4f-kube-api-access-vlnqx\") pod \"cert-manager-webhook-5dbbbf75cc-pvh7s\" (UID: \"31ccc096-b457-491c-a8cd-408865baaa4f\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-pvh7s"
Oct 30 19:10:49.111530 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.111436    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31ccc096-b457-491c-a8cd-408865baaa4f-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-pvh7s\" (UID: \"31ccc096-b457-491c-a8cd-408865baaa4f\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-pvh7s"
Oct 30 19:10:49.212821 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.212800    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlnqx\" (UniqueName: \"kubernetes.io/projected/31ccc096-b457-491c-a8cd-408865baaa4f-kube-api-access-vlnqx\") pod \"cert-manager-webhook-5dbbbf75cc-pvh7s\" (UID: \"31ccc096-b457-491c-a8cd-408865baaa4f\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-pvh7s"
Oct 30 19:10:49.212913 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.212830    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31ccc096-b457-491c-a8cd-408865baaa4f-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-pvh7s\" (UID: \"31ccc096-b457-491c-a8cd-408865baaa4f\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-pvh7s"
Oct 30 19:10:49.230471 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.230450    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlnqx\" (UniqueName: \"kubernetes.io/projected/31ccc096-b457-491c-a8cd-408865baaa4f-kube-api-access-vlnqx\") pod \"cert-manager-webhook-5dbbbf75cc-pvh7s\" (UID: \"31ccc096-b457-491c-a8cd-408865baaa4f\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-pvh7s"
Oct 30 19:10:49.230585 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.230565    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/31ccc096-b457-491c-a8cd-408865baaa4f-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-pvh7s\" (UID: \"31ccc096-b457-491c-a8cd-408865baaa4f\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-pvh7s"
Oct 30 19:10:49.350050 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.350026    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-pvh7s"
Oct 30 19:10:49.640491 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.640469    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-pvh7s"]
Oct 30 19:10:49.642679 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:10:49.642654    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31ccc096_b457_491c_a8cd_408865baaa4f.slice/crio-9665d87e056b9c11da2e46d1928dd2164492b9c794438423a6d57e5e0b046212 WatchSource:0}: Error finding container 9665d87e056b9c11da2e46d1928dd2164492b9c794438423a6d57e5e0b046212: Status 404 returned error can't find the container with id 9665d87e056b9c11da2e46d1928dd2164492b9c794438423a6d57e5e0b046212
Oct 30 19:10:49.765534 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.765508    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s"]
Oct 30 19:10:49.797999 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.797976    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s"]
Oct 30 19:10:49.798092 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.798064    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s"
Oct 30 19:10:49.801707 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.801685    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 19:10:49.802054 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.802039    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"61b1ff68c7255ee5c5329fe19a5d96293a9578-ce91a85d-dockercfg-k72kl\""
Oct 30 19:10:49.802128 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.802097    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 19:10:49.817992 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.817973    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98b2l\" (UniqueName: \"kubernetes.io/projected/9d2fc971-822f-4cea-9643-c59bfe01419e-kube-api-access-98b2l\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s\" (UID: \"9d2fc971-822f-4cea-9643-c59bfe01419e\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s"
Oct 30 19:10:49.818076 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.818005    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d2fc971-822f-4cea-9643-c59bfe01419e-bundle\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s\" (UID: \"9d2fc971-822f-4cea-9643-c59bfe01419e\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s"
Oct 30 19:10:49.818076 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.818028    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d2fc971-822f-4cea-9643-c59bfe01419e-util\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s\" (UID: \"9d2fc971-822f-4cea-9643-c59bfe01419e\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s"
Oct 30 19:10:49.919485 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.919429    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98b2l\" (UniqueName: \"kubernetes.io/projected/9d2fc971-822f-4cea-9643-c59bfe01419e-kube-api-access-98b2l\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s\" (UID: \"9d2fc971-822f-4cea-9643-c59bfe01419e\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s"
Oct 30 19:10:49.919485 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.919461    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d2fc971-822f-4cea-9643-c59bfe01419e-bundle\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s\" (UID: \"9d2fc971-822f-4cea-9643-c59bfe01419e\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s"
Oct 30 19:10:49.919601 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.919486    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d2fc971-822f-4cea-9643-c59bfe01419e-util\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s\" (UID: \"9d2fc971-822f-4cea-9643-c59bfe01419e\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s"
Oct 30 19:10:49.919754 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.919741    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d2fc971-822f-4cea-9643-c59bfe01419e-util\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s\" (UID: \"9d2fc971-822f-4cea-9643-c59bfe01419e\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s"
Oct 30 19:10:49.919876 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.919857    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d2fc971-822f-4cea-9643-c59bfe01419e-bundle\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s\" (UID: \"9d2fc971-822f-4cea-9643-c59bfe01419e\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s"
Oct 30 19:10:49.934227 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:49.934206    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98b2l\" (UniqueName: \"kubernetes.io/projected/9d2fc971-822f-4cea-9643-c59bfe01419e-kube-api-access-98b2l\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s\" (UID: \"9d2fc971-822f-4cea-9643-c59bfe01419e\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s"
Oct 30 19:10:50.149137 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:50.149113    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s"
Oct 30 19:10:50.431868 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:50.431790    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-pvh7s" event={"ID":"31ccc096-b457-491c-a8cd-408865baaa4f","Type":"ContainerStarted","Data":"9665d87e056b9c11da2e46d1928dd2164492b9c794438423a6d57e5e0b046212"}
Oct 30 19:10:50.438027 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:50.438000    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s"]
Oct 30 19:10:50.441305 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:10:50.441279    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d2fc971_822f_4cea_9643_c59bfe01419e.slice/crio-2aa11764329800b18574d619a4da07505cfa1108a24cb773769d69f265ec723d WatchSource:0}: Error finding container 2aa11764329800b18574d619a4da07505cfa1108a24cb773769d69f265ec723d: Status 404 returned error can't find the container with id 2aa11764329800b18574d619a4da07505cfa1108a24cb773769d69f265ec723d
Oct 30 19:10:51.437558 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:51.437526    4524 generic.go:356] "Generic (PLEG): container finished" podID="9d2fc971-822f-4cea-9643-c59bfe01419e" containerID="158a3b8fde2ff0562648c513d667010660d9bc433c42ad5ba26269271c47ca94" exitCode=0
Oct 30 19:10:51.438030 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:51.437573    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s" event={"ID":"9d2fc971-822f-4cea-9643-c59bfe01419e","Type":"ContainerDied","Data":"158a3b8fde2ff0562648c513d667010660d9bc433c42ad5ba26269271c47ca94"}
Oct 30 19:10:51.438030 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:51.437643    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s" event={"ID":"9d2fc971-822f-4cea-9643-c59bfe01419e","Type":"ContainerStarted","Data":"2aa11764329800b18574d619a4da07505cfa1108a24cb773769d69f265ec723d"}
Oct 30 19:10:53.449418 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:53.449380    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-pvh7s" event={"ID":"31ccc096-b457-491c-a8cd-408865baaa4f","Type":"ContainerStarted","Data":"280055eaead9e25f2cf8b80128b7f1da4aa621580ccd2b0ca071d2f8f4ce4e10"}
Oct 30 19:10:53.449968 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:53.449457    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-pvh7s"
Oct 30 19:10:53.466703 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:53.466622    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-pvh7s" podStartSLOduration=2.628037768 podStartE2EDuration="5.46660491s" totalImagesPullingTime="2.838567142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:10:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:10:53.466330913 +0000 UTC m=+285.207322005" watchObservedRunningTime="2026-10-30 19:10:53.46660491 +0000 UTC m=+285.207595967"
Oct 30 19:10:54.454426 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:54.454391    4524 generic.go:356] "Generic (PLEG): container finished" podID="9d2fc971-822f-4cea-9643-c59bfe01419e" containerID="a05412c70de9b73e91d00278c843ff53e3c32737c6feed65960ae48c7a37e6ea" exitCode=0
Oct 30 19:10:54.454917 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:54.454473    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s" event={"ID":"9d2fc971-822f-4cea-9643-c59bfe01419e","Type":"ContainerDied","Data":"a05412c70de9b73e91d00278c843ff53e3c32737c6feed65960ae48c7a37e6ea"}
Oct 30 19:10:55.460339 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:55.460303    4524 generic.go:356] "Generic (PLEG): container finished" podID="9d2fc971-822f-4cea-9643-c59bfe01419e" containerID="c1b6a16fee916f28ace9b37c948a3edf2cb962eee617935e713522fff6b8aecb" exitCode=0
Oct 30 19:10:55.460757 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:55.460336    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s" event={"ID":"9d2fc971-822f-4cea-9643-c59bfe01419e","Type":"ContainerDied","Data":"c1b6a16fee916f28ace9b37c948a3edf2cb962eee617935e713522fff6b8aecb"}
Oct 30 19:10:56.584313 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:56.584290    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s"
Oct 30 19:10:56.669211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:56.669182    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d2fc971-822f-4cea-9643-c59bfe01419e-bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d2fc971-822f-4cea-9643-c59bfe01419e-bundle\") pod \"9d2fc971-822f-4cea-9643-c59bfe01419e\" (UID: \"9d2fc971-822f-4cea-9643-c59bfe01419e\") "
Oct 30 19:10:56.669340 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:56.669227    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d2fc971-822f-4cea-9643-c59bfe01419e-kube-api-access-98b2l\" (UniqueName: \"kubernetes.io/projected/9d2fc971-822f-4cea-9643-c59bfe01419e-kube-api-access-98b2l\") pod \"9d2fc971-822f-4cea-9643-c59bfe01419e\" (UID: \"9d2fc971-822f-4cea-9643-c59bfe01419e\") "
Oct 30 19:10:56.669340 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:56.669266    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d2fc971-822f-4cea-9643-c59bfe01419e-util\" (UniqueName: \"kubernetes.io/empty-dir/9d2fc971-822f-4cea-9643-c59bfe01419e-util\") pod \"9d2fc971-822f-4cea-9643-c59bfe01419e\" (UID: \"9d2fc971-822f-4cea-9643-c59bfe01419e\") "
Oct 30 19:10:56.669591 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:56.669565    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d2fc971-822f-4cea-9643-c59bfe01419e-bundle" pod "9d2fc971-822f-4cea-9643-c59bfe01419e" (UID: "9d2fc971-822f-4cea-9643-c59bfe01419e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:10:56.671270 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:56.671242    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d2fc971-822f-4cea-9643-c59bfe01419e-kube-api-access-98b2l" pod "9d2fc971-822f-4cea-9643-c59bfe01419e" (UID: "9d2fc971-822f-4cea-9643-c59bfe01419e"). InnerVolumeSpecName "kube-api-access-98b2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:10:56.675729 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:56.675706    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d2fc971-822f-4cea-9643-c59bfe01419e-util" pod "9d2fc971-822f-4cea-9643-c59bfe01419e" (UID: "9d2fc971-822f-4cea-9643-c59bfe01419e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:10:56.769860 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:56.769809    4524 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d2fc971-822f-4cea-9643-c59bfe01419e-util\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:56.769860 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:56.769830    4524 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d2fc971-822f-4cea-9643-c59bfe01419e-bundle\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:56.769860 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:56.769839    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98b2l\" (UniqueName: \"kubernetes.io/projected/9d2fc971-822f-4cea-9643-c59bfe01419e-kube-api-access-98b2l\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:10:57.469015 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:57.468977    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s" event={"ID":"9d2fc971-822f-4cea-9643-c59bfe01419e","Type":"ContainerDied","Data":"2aa11764329800b18574d619a4da07505cfa1108a24cb773769d69f265ec723d"}
Oct 30 19:10:57.469015 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:57.469000    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85mm82s"
Oct 30 19:10:57.469272 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:57.469026    4524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2aa11764329800b18574d619a4da07505cfa1108a24cb773769d69f265ec723d"
Oct 30 19:10:59.456370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:10:59.456343    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-pvh7s"
Oct 30 19:11:02.767624 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:02.767588    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-rf96v"]
Oct 30 19:11:02.768115 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:02.767971    4524 cpu_manager.go:414] "containerMap: removing container" podUID="9d2fc971-822f-4cea-9643-c59bfe01419e" containerName="pull"
Oct 30 19:11:02.768115 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:02.767989    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d2fc971-822f-4cea-9643-c59bfe01419e" containerName="pull"
Oct 30 19:11:02.768115 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:02.768009    4524 cpu_manager.go:414] "containerMap: removing container" podUID="9d2fc971-822f-4cea-9643-c59bfe01419e" containerName="extract"
Oct 30 19:11:02.768115 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:02.768016    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d2fc971-822f-4cea-9643-c59bfe01419e" containerName="extract"
Oct 30 19:11:02.768115 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:02.768048    4524 cpu_manager.go:414] "containerMap: removing container" podUID="9d2fc971-822f-4cea-9643-c59bfe01419e" containerName="util"
Oct 30 19:11:02.768115 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:02.768055    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d2fc971-822f-4cea-9643-c59bfe01419e" containerName="util"
Oct 30 19:11:02.768115 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:02.768116    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d2fc971-822f-4cea-9643-c59bfe01419e" containerName="extract"
Oct 30 19:11:02.770061 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:02.770041    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-8465994787-rf96v"
Oct 30 19:11:02.773333 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:02.773311    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Oct 30 19:11:02.773481 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:02.773465    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-nwlf5\""
Oct 30 19:11:02.773545 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:02.773505    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Oct 30 19:11:02.780935 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:02.780916    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-rf96v"]
Oct 30 19:11:02.812703 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:02.812676    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e802a83-9018-428e-85b5-f94b9016ed90-tmp\") pod \"openshift-lws-operator-8465994787-rf96v\" (UID: \"8e802a83-9018-428e-85b5-f94b9016ed90\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-rf96v"
Oct 30 19:11:02.812815 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:02.812707    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7sgl\" (UniqueName: \"kubernetes.io/projected/8e802a83-9018-428e-85b5-f94b9016ed90-kube-api-access-n7sgl\") pod \"openshift-lws-operator-8465994787-rf96v\" (UID: \"8e802a83-9018-428e-85b5-f94b9016ed90\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-rf96v"
Oct 30 19:11:02.913773 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:02.913743    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e802a83-9018-428e-85b5-f94b9016ed90-tmp\") pod \"openshift-lws-operator-8465994787-rf96v\" (UID: \"8e802a83-9018-428e-85b5-f94b9016ed90\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-rf96v"
Oct 30 19:11:02.913891 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:02.913777    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7sgl\" (UniqueName: \"kubernetes.io/projected/8e802a83-9018-428e-85b5-f94b9016ed90-kube-api-access-n7sgl\") pod \"openshift-lws-operator-8465994787-rf96v\" (UID: \"8e802a83-9018-428e-85b5-f94b9016ed90\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-rf96v"
Oct 30 19:11:02.914224 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:02.914209    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e802a83-9018-428e-85b5-f94b9016ed90-tmp\") pod \"openshift-lws-operator-8465994787-rf96v\" (UID: \"8e802a83-9018-428e-85b5-f94b9016ed90\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-rf96v"
Oct 30 19:11:02.925676 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:02.925647    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7sgl\" (UniqueName: \"kubernetes.io/projected/8e802a83-9018-428e-85b5-f94b9016ed90-kube-api-access-n7sgl\") pod \"openshift-lws-operator-8465994787-rf96v\" (UID: \"8e802a83-9018-428e-85b5-f94b9016ed90\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-rf96v"
Oct 30 19:11:03.080289 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:03.080255    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-8465994787-rf96v"
Oct 30 19:11:03.351574 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:03.351502    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-rf96v"]
Oct 30 19:11:03.354488 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:11:03.354459    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e802a83_9018_428e_85b5_f94b9016ed90.slice/crio-0e4f4457374aa509261ecb67122375833d4a164cf82e8e7aea6ebcfd4bf08ae2 WatchSource:0}: Error finding container 0e4f4457374aa509261ecb67122375833d4a164cf82e8e7aea6ebcfd4bf08ae2: Status 404 returned error can't find the container with id 0e4f4457374aa509261ecb67122375833d4a164cf82e8e7aea6ebcfd4bf08ae2
Oct 30 19:11:03.491517 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:03.491485    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-8465994787-rf96v" event={"ID":"8e802a83-9018-428e-85b5-f94b9016ed90","Type":"ContainerStarted","Data":"0e4f4457374aa509261ecb67122375833d4a164cf82e8e7aea6ebcfd4bf08ae2"}
Oct 30 19:11:15.623729 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:11:15.623689    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:11:18.547961 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:18.547927    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-8465994787-rf96v" event={"ID":"8e802a83-9018-428e-85b5-f94b9016ed90","Type":"ContainerStarted","Data":"c0c326c3c6b7229ea32b7a28e695da031cac5249b00fd2ae957ddd76f1cb7590"}
Oct 30 19:11:18.565807 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:18.565764    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-8465994787-rf96v" podStartSLOduration=2.344893156 podStartE2EDuration="16.565753109s" totalImagesPullingTime="14.220859953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:18.565463662 +0000 UTC m=+310.306454741" watchObservedRunningTime="2026-10-30 19:11:18.565753109 +0000 UTC m=+310.306744188"
Oct 30 19:11:21.357058 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:21.356984    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv"]
Oct 30 19:11:21.359179 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:21.359151    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv"
Oct 30 19:11:21.362325 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:21.362305    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 19:11:21.362911 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:21.362896    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 19:11:21.362973 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:21.362933    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-hwj5r\""
Oct 30 19:11:21.370488 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:21.370469    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv"]
Oct 30 19:11:21.445934 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:21.445909    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c60710ea-22c4-4291-8501-087304f3bf28-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv\" (UID: \"c60710ea-22c4-4291-8501-087304f3bf28\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv"
Oct 30 19:11:21.446048 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:21.445969    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgl2l\" (UniqueName: \"kubernetes.io/projected/c60710ea-22c4-4291-8501-087304f3bf28-kube-api-access-bgl2l\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv\" (UID: \"c60710ea-22c4-4291-8501-087304f3bf28\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv"
Oct 30 19:11:21.446118 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:21.446103    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c60710ea-22c4-4291-8501-087304f3bf28-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv\" (UID: \"c60710ea-22c4-4291-8501-087304f3bf28\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv"
Oct 30 19:11:21.546675 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:21.546651    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c60710ea-22c4-4291-8501-087304f3bf28-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv\" (UID: \"c60710ea-22c4-4291-8501-087304f3bf28\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv"
Oct 30 19:11:21.546801 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:21.546684    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c60710ea-22c4-4291-8501-087304f3bf28-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv\" (UID: \"c60710ea-22c4-4291-8501-087304f3bf28\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv"
Oct 30 19:11:21.546801 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:21.546717    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgl2l\" (UniqueName: \"kubernetes.io/projected/c60710ea-22c4-4291-8501-087304f3bf28-kube-api-access-bgl2l\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv\" (UID: \"c60710ea-22c4-4291-8501-087304f3bf28\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv"
Oct 30 19:11:21.547155 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:21.547136    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c60710ea-22c4-4291-8501-087304f3bf28-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv\" (UID: \"c60710ea-22c4-4291-8501-087304f3bf28\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv"
Oct 30 19:11:21.547229 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:21.547139    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c60710ea-22c4-4291-8501-087304f3bf28-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv\" (UID: \"c60710ea-22c4-4291-8501-087304f3bf28\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv"
Oct 30 19:11:21.561390 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:21.561365    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgl2l\" (UniqueName: \"kubernetes.io/projected/c60710ea-22c4-4291-8501-087304f3bf28-kube-api-access-bgl2l\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv\" (UID: \"c60710ea-22c4-4291-8501-087304f3bf28\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv"
Oct 30 19:11:21.672231 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:21.672149    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv"
Oct 30 19:11:21.941400 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:21.941331    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv"]
Oct 30 19:11:21.944207 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:11:21.944157    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc60710ea_22c4_4291_8501_087304f3bf28.slice/crio-593c1a311ffc0dcff5531c731ba4af57a9efd862c6243df6e8955dff0d2805fd WatchSource:0}: Error finding container 593c1a311ffc0dcff5531c731ba4af57a9efd862c6243df6e8955dff0d2805fd: Status 404 returned error can't find the container with id 593c1a311ffc0dcff5531c731ba4af57a9efd862c6243df6e8955dff0d2805fd
Oct 30 19:11:22.564545 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:22.564513    4524 generic.go:356] "Generic (PLEG): container finished" podID="c60710ea-22c4-4291-8501-087304f3bf28" containerID="755933a5ebdc6abac6e946aaa0b2cbe017a92584ce8f7bdb95ad868f6f7f80f2" exitCode=0
Oct 30 19:11:22.564975 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:22.564561    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv" event={"ID":"c60710ea-22c4-4291-8501-087304f3bf28","Type":"ContainerDied","Data":"755933a5ebdc6abac6e946aaa0b2cbe017a92584ce8f7bdb95ad868f6f7f80f2"}
Oct 30 19:11:22.564975 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:22.564609    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv" event={"ID":"c60710ea-22c4-4291-8501-087304f3bf28","Type":"ContainerStarted","Data":"593c1a311ffc0dcff5531c731ba4af57a9efd862c6243df6e8955dff0d2805fd"}
Oct 30 19:11:22.565059 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:22.565043    4524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:11:23.570147 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:23.570113    4524 generic.go:356] "Generic (PLEG): container finished" podID="c60710ea-22c4-4291-8501-087304f3bf28" containerID="6be95980bfd618d152858c8525787995302b47351a696509f1259e9265857135" exitCode=0
Oct 30 19:11:23.570652 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:23.570187    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv" event={"ID":"c60710ea-22c4-4291-8501-087304f3bf28","Type":"ContainerDied","Data":"6be95980bfd618d152858c8525787995302b47351a696509f1259e9265857135"}
Oct 30 19:11:24.575811 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:24.575779    4524 generic.go:356] "Generic (PLEG): container finished" podID="c60710ea-22c4-4291-8501-087304f3bf28" containerID="1374e062790a3326e08447ca0896cf06f3803269820847f065ad9be30b168502" exitCode=0
Oct 30 19:11:24.576208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:24.575863    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv" event={"ID":"c60710ea-22c4-4291-8501-087304f3bf28","Type":"ContainerDied","Data":"1374e062790a3326e08447ca0896cf06f3803269820847f065ad9be30b168502"}
Oct 30 19:11:25.706715 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:25.706691    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv"
Oct 30 19:11:25.776031 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:25.776002    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c60710ea-22c4-4291-8501-087304f3bf28-kube-api-access-bgl2l\" (UniqueName: \"kubernetes.io/projected/c60710ea-22c4-4291-8501-087304f3bf28-kube-api-access-bgl2l\") pod \"c60710ea-22c4-4291-8501-087304f3bf28\" (UID: \"c60710ea-22c4-4291-8501-087304f3bf28\") "
Oct 30 19:11:25.776205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:25.776038    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c60710ea-22c4-4291-8501-087304f3bf28-util\" (UniqueName: \"kubernetes.io/empty-dir/c60710ea-22c4-4291-8501-087304f3bf28-util\") pod \"c60710ea-22c4-4291-8501-087304f3bf28\" (UID: \"c60710ea-22c4-4291-8501-087304f3bf28\") "
Oct 30 19:11:25.776205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:25.776083    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c60710ea-22c4-4291-8501-087304f3bf28-bundle\" (UniqueName: \"kubernetes.io/empty-dir/c60710ea-22c4-4291-8501-087304f3bf28-bundle\") pod \"c60710ea-22c4-4291-8501-087304f3bf28\" (UID: \"c60710ea-22c4-4291-8501-087304f3bf28\") "
Oct 30 19:11:25.776693 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:25.776663    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c60710ea-22c4-4291-8501-087304f3bf28-bundle" pod "c60710ea-22c4-4291-8501-087304f3bf28" (UID: "c60710ea-22c4-4291-8501-087304f3bf28"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:11:25.778125 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:25.778107    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c60710ea-22c4-4291-8501-087304f3bf28-kube-api-access-bgl2l" pod "c60710ea-22c4-4291-8501-087304f3bf28" (UID: "c60710ea-22c4-4291-8501-087304f3bf28"). InnerVolumeSpecName "kube-api-access-bgl2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:11:25.783576 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:25.783550    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c60710ea-22c4-4291-8501-087304f3bf28-util" pod "c60710ea-22c4-4291-8501-087304f3bf28" (UID: "c60710ea-22c4-4291-8501-087304f3bf28"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:11:25.876528 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:25.876463    4524 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c60710ea-22c4-4291-8501-087304f3bf28-bundle\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:11:25.876528 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:25.876488    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgl2l\" (UniqueName: \"kubernetes.io/projected/c60710ea-22c4-4291-8501-087304f3bf28-kube-api-access-bgl2l\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:11:25.876528 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:25.876497    4524 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c60710ea-22c4-4291-8501-087304f3bf28-util\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:11:26.584597 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:26.584569    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv"
Oct 30 19:11:26.584597 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:26.584575    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jwqqv" event={"ID":"c60710ea-22c4-4291-8501-087304f3bf28","Type":"ContainerDied","Data":"593c1a311ffc0dcff5531c731ba4af57a9efd862c6243df6e8955dff0d2805fd"}
Oct 30 19:11:26.584802 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:26.584616    4524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="593c1a311ffc0dcff5531c731ba4af57a9efd862c6243df6e8955dff0d2805fd"
Oct 30 19:11:37.135158 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.135123    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"]
Oct 30 19:11:37.135553 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.135431    4524 cpu_manager.go:414] "containerMap: removing container" podUID="c60710ea-22c4-4291-8501-087304f3bf28" containerName="util"
Oct 30 19:11:37.135553 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.135443    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c60710ea-22c4-4291-8501-087304f3bf28" containerName="util"
Oct 30 19:11:37.135553 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.135456    4524 cpu_manager.go:414] "containerMap: removing container" podUID="c60710ea-22c4-4291-8501-087304f3bf28" containerName="pull"
Oct 30 19:11:37.135553 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.135461    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c60710ea-22c4-4291-8501-087304f3bf28" containerName="pull"
Oct 30 19:11:37.135553 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.135479    4524 cpu_manager.go:414] "containerMap: removing container" podUID="c60710ea-22c4-4291-8501-087304f3bf28" containerName="extract"
Oct 30 19:11:37.135553 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.135486    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c60710ea-22c4-4291-8501-087304f3bf28" containerName="extract"
Oct 30 19:11:37.135553 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.135531    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="c60710ea-22c4-4291-8501-087304f3bf28" containerName="extract"
Oct 30 19:11:37.137694 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.137677    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:37.150760 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.150738    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 19:11:37.150939 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.150888    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 19:11:37.151269 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.151256    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 19:11:37.151511 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.151499    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 19:11:37.151662 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.151649    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-v5t6d\""
Oct 30 19:11:37.152130 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.152119    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-metrics-tls\""
Oct 30 19:11:37.185738 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.185712    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"]
Oct 30 19:11:37.263334 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.263299    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8bjm\" (UniqueName: \"kubernetes.io/projected/8b72130d-260a-4550-a9b6-3caeedcedb71-kube-api-access-v8bjm\") pod \"opendatahub-operator-controller-manager-7c94c66984-zfkdh\" (UID: \"8b72130d-260a-4550-a9b6-3caeedcedb71\") " pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:37.263487 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.263371    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b72130d-260a-4550-a9b6-3caeedcedb71-metrics-cert\") pod \"opendatahub-operator-controller-manager-7c94c66984-zfkdh\" (UID: \"8b72130d-260a-4550-a9b6-3caeedcedb71\") " pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:37.263487 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.263441    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b72130d-260a-4550-a9b6-3caeedcedb71-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c94c66984-zfkdh\" (UID: \"8b72130d-260a-4550-a9b6-3caeedcedb71\") " pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:37.263487 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.263477    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8b72130d-260a-4550-a9b6-3caeedcedb71-manifests\") pod \"opendatahub-operator-controller-manager-7c94c66984-zfkdh\" (UID: \"8b72130d-260a-4550-a9b6-3caeedcedb71\") " pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:37.263616 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.263497    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b72130d-260a-4550-a9b6-3caeedcedb71-tmp\") pod \"opendatahub-operator-controller-manager-7c94c66984-zfkdh\" (UID: \"8b72130d-260a-4550-a9b6-3caeedcedb71\") " pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:37.263616 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.263569    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8b72130d-260a-4550-a9b6-3caeedcedb71-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c94c66984-zfkdh\" (UID: \"8b72130d-260a-4550-a9b6-3caeedcedb71\") " pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:37.364089 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.364053    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8bjm\" (UniqueName: \"kubernetes.io/projected/8b72130d-260a-4550-a9b6-3caeedcedb71-kube-api-access-v8bjm\") pod \"opendatahub-operator-controller-manager-7c94c66984-zfkdh\" (UID: \"8b72130d-260a-4550-a9b6-3caeedcedb71\") " pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:37.364270 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.364107    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b72130d-260a-4550-a9b6-3caeedcedb71-metrics-cert\") pod \"opendatahub-operator-controller-manager-7c94c66984-zfkdh\" (UID: \"8b72130d-260a-4550-a9b6-3caeedcedb71\") " pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:37.364270 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.364128    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b72130d-260a-4550-a9b6-3caeedcedb71-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c94c66984-zfkdh\" (UID: \"8b72130d-260a-4550-a9b6-3caeedcedb71\") " pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:37.364270 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.364159    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8b72130d-260a-4550-a9b6-3caeedcedb71-manifests\") pod \"opendatahub-operator-controller-manager-7c94c66984-zfkdh\" (UID: \"8b72130d-260a-4550-a9b6-3caeedcedb71\") " pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:37.364270 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.364204    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b72130d-260a-4550-a9b6-3caeedcedb71-tmp\") pod \"opendatahub-operator-controller-manager-7c94c66984-zfkdh\" (UID: \"8b72130d-260a-4550-a9b6-3caeedcedb71\") " pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:37.364270 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.364234    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8b72130d-260a-4550-a9b6-3caeedcedb71-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c94c66984-zfkdh\" (UID: \"8b72130d-260a-4550-a9b6-3caeedcedb71\") " pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:37.364654 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.364623    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8b72130d-260a-4550-a9b6-3caeedcedb71-manifests\") pod \"opendatahub-operator-controller-manager-7c94c66984-zfkdh\" (UID: \"8b72130d-260a-4550-a9b6-3caeedcedb71\") " pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:37.366638 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.366613    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8b72130d-260a-4550-a9b6-3caeedcedb71-metrics-cert\") pod \"opendatahub-operator-controller-manager-7c94c66984-zfkdh\" (UID: \"8b72130d-260a-4550-a9b6-3caeedcedb71\") " pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:37.366737 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.366644    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b72130d-260a-4550-a9b6-3caeedcedb71-tmp\") pod \"opendatahub-operator-controller-manager-7c94c66984-zfkdh\" (UID: \"8b72130d-260a-4550-a9b6-3caeedcedb71\") " pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:37.366737 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.366715    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8b72130d-260a-4550-a9b6-3caeedcedb71-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c94c66984-zfkdh\" (UID: \"8b72130d-260a-4550-a9b6-3caeedcedb71\") " pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:37.366802 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.366740    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8b72130d-260a-4550-a9b6-3caeedcedb71-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c94c66984-zfkdh\" (UID: \"8b72130d-260a-4550-a9b6-3caeedcedb71\") " pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:37.375931 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.375907    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8bjm\" (UniqueName: \"kubernetes.io/projected/8b72130d-260a-4550-a9b6-3caeedcedb71-kube-api-access-v8bjm\") pod \"opendatahub-operator-controller-manager-7c94c66984-zfkdh\" (UID: \"8b72130d-260a-4550-a9b6-3caeedcedb71\") " pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:37.450283 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.450201    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:37.749389 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:37.749300    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"]
Oct 30 19:11:37.752262 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:11:37.752231    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b72130d_260a_4550_a9b6_3caeedcedb71.slice/crio-3d9655f1358d1bebed5cce06ed9251f1ba3d631c73f579dfc9e51320f38727b2 WatchSource:0}: Error finding container 3d9655f1358d1bebed5cce06ed9251f1ba3d631c73f579dfc9e51320f38727b2: Status 404 returned error can't find the container with id 3d9655f1358d1bebed5cce06ed9251f1ba3d631c73f579dfc9e51320f38727b2
Oct 30 19:11:38.632718 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:38.632636    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh" event={"ID":"8b72130d-260a-4550-a9b6-3caeedcedb71","Type":"ContainerStarted","Data":"3d9655f1358d1bebed5cce06ed9251f1ba3d631c73f579dfc9e51320f38727b2"}
Oct 30 19:11:40.646348 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:40.646318    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh" event={"ID":"8b72130d-260a-4550-a9b6-3caeedcedb71","Type":"ContainerStarted","Data":"e5fc8166d5149f98c0e1ac2c893255580cef0717a306ddfa570adac0b3e92719"}
Oct 30 19:11:41.652350 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:41.652311    4524 generic.go:356] "Generic (PLEG): container finished" podID="8b72130d-260a-4550-a9b6-3caeedcedb71" containerID="e5fc8166d5149f98c0e1ac2c893255580cef0717a306ddfa570adac0b3e92719" exitCode=0
Oct 30 19:11:41.652801 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:41.652404    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh" event={"ID":"8b72130d-260a-4550-a9b6-3caeedcedb71","Type":"ContainerDied","Data":"e5fc8166d5149f98c0e1ac2c893255580cef0717a306ddfa570adac0b3e92719"}
Oct 30 19:11:42.658740 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:42.658707    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh" event={"ID":"8b72130d-260a-4550-a9b6-3caeedcedb71","Type":"ContainerStarted","Data":"dcbba9a13eefd9c5fcd6332de2186f044de97b8353b73f0bb25350b2599508ea"}
Oct 30 19:11:42.659133 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:42.658835    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:42.708014 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:42.707968    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh" podStartSLOduration=1.958674029 podStartE2EDuration="5.707953897s" totalImagesPullingTime="3.295662355s" totalInitContainerRuntime="453.617513ms" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:42.706214117 +0000 UTC m=+334.447205197" watchObservedRunningTime="2026-10-30 19:11:42.707953897 +0000 UTC m=+334.448944976"
Oct 30 19:11:45.623936 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:11:45.623880    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:11:53.664995 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:53.664968    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7c94c66984-zfkdh"
Oct 30 19:11:56.021847 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:56.021811    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-678bf485d7-kpxzm"]
Oct 30 19:11:56.025086 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:56.025070    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-678bf485d7-kpxzm"
Oct 30 19:11:56.029771 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:56.029744    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 30 19:11:56.029771 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:56.029757    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 19:11:56.029771 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:56.029766    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 19:11:56.029996 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:56.029755    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 19:11:56.029996 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:56.029845    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-6zhvc\""
Oct 30 19:11:56.041672 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:56.041648    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-678bf485d7-kpxzm"]
Oct 30 19:11:56.208288 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:56.208257    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2-tmp\") pod \"kube-auth-proxy-678bf485d7-kpxzm\" (UID: \"b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2\") " pod="openshift-ingress/kube-auth-proxy-678bf485d7-kpxzm"
Oct 30 19:11:56.208456 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:56.208304    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tss2n\" (UniqueName: \"kubernetes.io/projected/b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2-kube-api-access-tss2n\") pod \"kube-auth-proxy-678bf485d7-kpxzm\" (UID: \"b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2\") " pod="openshift-ingress/kube-auth-proxy-678bf485d7-kpxzm"
Oct 30 19:11:56.208456 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:56.208445    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2-tls-certs\") pod \"kube-auth-proxy-678bf485d7-kpxzm\" (UID: \"b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2\") " pod="openshift-ingress/kube-auth-proxy-678bf485d7-kpxzm"
Oct 30 19:11:56.309091 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:56.309027    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2-tls-certs\") pod \"kube-auth-proxy-678bf485d7-kpxzm\" (UID: \"b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2\") " pod="openshift-ingress/kube-auth-proxy-678bf485d7-kpxzm"
Oct 30 19:11:56.309091 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:56.309058    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2-tmp\") pod \"kube-auth-proxy-678bf485d7-kpxzm\" (UID: \"b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2\") " pod="openshift-ingress/kube-auth-proxy-678bf485d7-kpxzm"
Oct 30 19:11:56.309091 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:56.309087    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tss2n\" (UniqueName: \"kubernetes.io/projected/b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2-kube-api-access-tss2n\") pod \"kube-auth-proxy-678bf485d7-kpxzm\" (UID: \"b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2\") " pod="openshift-ingress/kube-auth-proxy-678bf485d7-kpxzm"
Oct 30 19:11:56.311302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:56.311276    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2-tmp\") pod \"kube-auth-proxy-678bf485d7-kpxzm\" (UID: \"b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2\") " pod="openshift-ingress/kube-auth-proxy-678bf485d7-kpxzm"
Oct 30 19:11:56.311434 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:56.311416    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2-tls-certs\") pod \"kube-auth-proxy-678bf485d7-kpxzm\" (UID: \"b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2\") " pod="openshift-ingress/kube-auth-proxy-678bf485d7-kpxzm"
Oct 30 19:11:56.323725 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:56.323703    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tss2n\" (UniqueName: \"kubernetes.io/projected/b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2-kube-api-access-tss2n\") pod \"kube-auth-proxy-678bf485d7-kpxzm\" (UID: \"b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2\") " pod="openshift-ingress/kube-auth-proxy-678bf485d7-kpxzm"
Oct 30 19:11:56.336653 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:56.336634    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-678bf485d7-kpxzm"
Oct 30 19:11:56.607045 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:56.607019    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-678bf485d7-kpxzm"]
Oct 30 19:11:56.609773 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:11:56.609745    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb95fd459_3c9c_4a5a_9a4f_6c08116e6ca2.slice/crio-b5a75f2c45f5d7b682ec18aa93289eb48893bd85b3b305cad30f8e38feeae1de WatchSource:0}: Error finding container b5a75f2c45f5d7b682ec18aa93289eb48893bd85b3b305cad30f8e38feeae1de: Status 404 returned error can't find the container with id b5a75f2c45f5d7b682ec18aa93289eb48893bd85b3b305cad30f8e38feeae1de
Oct 30 19:11:56.711049 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:56.711021    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-678bf485d7-kpxzm" event={"ID":"b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2","Type":"ContainerStarted","Data":"b5a75f2c45f5d7b682ec18aa93289eb48893bd85b3b305cad30f8e38feeae1de"}
Oct 30 19:11:57.595270 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.595239    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-observability-584f99c788-p45zp"]
Oct 30 19:11:57.598677 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.598655    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-observability-584f99c788-p45zp"
Oct 30 19:11:57.606619 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.606597    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-observability-webhook-cert\""
Oct 30 19:11:57.606736 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.606648    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-observability-dockercfg-8ll9b\""
Oct 30 19:11:57.611201 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.611153    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-observability-584f99c788-p45zp"]
Oct 30 19:11:57.699747 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.699717    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"]
Oct 30 19:11:57.703126 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.703105    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.710366 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.710344    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 19:11:57.711981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.711964    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 19:11:57.712165 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.712121    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-642gd\""
Oct 30 19:11:57.712165 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.712127    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 19:11:57.712312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.712216    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 19:11:57.718716 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.718692    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"]
Oct 30 19:11:57.718997 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.718974    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnfhd\" (UniqueName: \"kubernetes.io/projected/bca23000-fcb5-41f7-9051-f5feaa096fe4-kube-api-access-mnfhd\") pod \"odh-observability-584f99c788-p45zp\" (UID: \"bca23000-fcb5-41f7-9051-f5feaa096fe4\") " pod="opendatahub/odh-observability-584f99c788-p45zp"
Oct 30 19:11:57.719077 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.719061    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/bca23000-fcb5-41f7-9051-f5feaa096fe4-webhook-certs\") pod \"odh-observability-584f99c788-p45zp\" (UID: \"bca23000-fcb5-41f7-9051-f5feaa096fe4\") " pod="opendatahub/odh-observability-584f99c788-p45zp"
Oct 30 19:11:57.819562 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.819523    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/19679075-db01-441b-a98b-1378cfe9ddbc-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.819562 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.819566    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/19679075-db01-441b-a98b-1378cfe9ddbc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.819804 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.819664    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.819804 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.819713    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/bca23000-fcb5-41f7-9051-f5feaa096fe4-webhook-certs\") pod \"odh-observability-584f99c788-p45zp\" (UID: \"bca23000-fcb5-41f7-9051-f5feaa096fe4\") " pod="opendatahub/odh-observability-584f99c788-p45zp"
Oct 30 19:11:57.819939 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.819821    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.819939 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:11:57.819838    4524 secret.go:189] Couldn't get secret opendatahub/odh-observability-webhook-cert: secret "odh-observability-webhook-cert" not found
Oct 30 19:11:57.819939 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:11:57.819899    4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bca23000-fcb5-41f7-9051-f5feaa096fe4-webhook-certs podName:bca23000-fcb5-41f7-9051-f5feaa096fe4 nodeName:}" failed. No retries permitted until 2026-10-30 19:11:58.319882424 +0000 UTC m=+350.060873488 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-certs" (UniqueName: "kubernetes.io/secret/bca23000-fcb5-41f7-9051-f5feaa096fe4-webhook-certs") pod "odh-observability-584f99c788-p45zp" (UID: "bca23000-fcb5-41f7-9051-f5feaa096fe4") : secret "odh-observability-webhook-cert" not found
Oct 30 19:11:57.819939 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.819917    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19679075-db01-441b-a98b-1378cfe9ddbc-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.820159 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.819965    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.820159 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.819993    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhqrn\" (UniqueName: \"kubernetes.io/projected/19679075-db01-441b-a98b-1378cfe9ddbc-kube-api-access-qhqrn\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.820159 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.820028    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnfhd\" (UniqueName: \"kubernetes.io/projected/bca23000-fcb5-41f7-9051-f5feaa096fe4-kube-api-access-mnfhd\") pod \"odh-observability-584f99c788-p45zp\" (UID: \"bca23000-fcb5-41f7-9051-f5feaa096fe4\") " pod="opendatahub/odh-observability-584f99c788-p45zp"
Oct 30 19:11:57.834518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.834470    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnfhd\" (UniqueName: \"kubernetes.io/projected/bca23000-fcb5-41f7-9051-f5feaa096fe4-kube-api-access-mnfhd\") pod \"odh-observability-584f99c788-p45zp\" (UID: \"bca23000-fcb5-41f7-9051-f5feaa096fe4\") " pod="opendatahub/odh-observability-584f99c788-p45zp"
Oct 30 19:11:57.920734 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.920668    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.920883 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.920736    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.920883 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.920763    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19679075-db01-441b-a98b-1378cfe9ddbc-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.920883 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.920804    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.920883 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.920839    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhqrn\" (UniqueName: \"kubernetes.io/projected/19679075-db01-441b-a98b-1378cfe9ddbc-kube-api-access-qhqrn\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.921096 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.920883    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/19679075-db01-441b-a98b-1378cfe9ddbc-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.921096 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.921058    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/19679075-db01-441b-a98b-1378cfe9ddbc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.921768 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.921736    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/19679075-db01-441b-a98b-1378cfe9ddbc-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.923191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.923149    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/19679075-db01-441b-a98b-1378cfe9ddbc-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.923429 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.923407    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.923633 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.923616    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.925812 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.925765    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.932281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.932248    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19679075-db01-441b-a98b-1378cfe9ddbc-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:57.932671 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:57.932648    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhqrn\" (UniqueName: \"kubernetes.io/projected/19679075-db01-441b-a98b-1378cfe9ddbc-kube-api-access-qhqrn\") pod \"istiod-openshift-gateway-7876bfcb74-4fwqk\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:58.016131 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:58.016101    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:11:58.305278 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:58.305192    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"]
Oct 30 19:11:58.310282 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:11:58.310257    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19679075_db01_441b_a98b_1378cfe9ddbc.slice/crio-a00767e439850575623cb54b43a44b20be00fe6305c3e95833da35ee482dd3d5 WatchSource:0}: Error finding container a00767e439850575623cb54b43a44b20be00fe6305c3e95833da35ee482dd3d5: Status 404 returned error can't find the container with id a00767e439850575623cb54b43a44b20be00fe6305c3e95833da35ee482dd3d5
Oct 30 19:11:58.325676 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:58.325653    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/bca23000-fcb5-41f7-9051-f5feaa096fe4-webhook-certs\") pod \"odh-observability-584f99c788-p45zp\" (UID: \"bca23000-fcb5-41f7-9051-f5feaa096fe4\") " pod="opendatahub/odh-observability-584f99c788-p45zp"
Oct 30 19:11:58.328500 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:58.328479    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/bca23000-fcb5-41f7-9051-f5feaa096fe4-webhook-certs\") pod \"odh-observability-584f99c788-p45zp\" (UID: \"bca23000-fcb5-41f7-9051-f5feaa096fe4\") " pod="opendatahub/odh-observability-584f99c788-p45zp"
Oct 30 19:11:58.515249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:58.515213    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-observability-584f99c788-p45zp"
Oct 30 19:11:58.722940 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:58.722906    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk" event={"ID":"19679075-db01-441b-a98b-1378cfe9ddbc","Type":"ContainerStarted","Data":"a00767e439850575623cb54b43a44b20be00fe6305c3e95833da35ee482dd3d5"}
Oct 30 19:11:58.791259 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:58.791215    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-observability-584f99c788-p45zp"]
Oct 30 19:11:58.795390 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:11:58.795360    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbca23000_fcb5_41f7_9051_f5feaa096fe4.slice/crio-d7ed2988dbfcf886900bdf22f78d1850f948ff488ca77b0977ede27f1728eb1d WatchSource:0}: Error finding container d7ed2988dbfcf886900bdf22f78d1850f948ff488ca77b0977ede27f1728eb1d: Status 404 returned error can't find the container with id d7ed2988dbfcf886900bdf22f78d1850f948ff488ca77b0977ede27f1728eb1d
Oct 30 19:11:59.732608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:59.732562    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-678bf485d7-kpxzm" event={"ID":"b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2","Type":"ContainerStarted","Data":"aedfd422a1cd1baf011cfd0267b6321f2282ec3952d306525eddff470f4758ca"}
Oct 30 19:11:59.734803 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:59.734778    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-observability-584f99c788-p45zp" event={"ID":"bca23000-fcb5-41f7-9051-f5feaa096fe4","Type":"ContainerStarted","Data":"d7ed2988dbfcf886900bdf22f78d1850f948ff488ca77b0977ede27f1728eb1d"}
Oct 30 19:11:59.751775 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:11:59.751724    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-678bf485d7-kpxzm" podStartSLOduration=1.5023961689999998 podStartE2EDuration="3.751705201s" totalImagesPullingTime="2.249309032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:59.749898039 +0000 UTC m=+351.490889117" watchObservedRunningTime="2026-10-30 19:11:59.751705201 +0000 UTC m=+351.492696307"
Oct 30 19:12:01.576616 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:01.576574    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:12:01.577033 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:01.576663    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:12:01.745081 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:01.745038    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-observability-584f99c788-p45zp" event={"ID":"bca23000-fcb5-41f7-9051-f5feaa096fe4","Type":"ContainerStarted","Data":"e83f926ac272f94f7835ac70ebf58f3c2974a298ab489887a6f12e605947c8b7"}
Oct 30 19:12:01.745401 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:01.745117    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-observability-584f99c788-p45zp"
Oct 30 19:12:01.746725 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:01.746699    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk" event={"ID":"19679075-db01-441b-a98b-1378cfe9ddbc","Type":"ContainerStarted","Data":"56ab84ca70e827ae323c054cb65e718016ed5ce5eb63ec20dab6c64cbbb2917a"}
Oct 30 19:12:01.746838 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:01.746828    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:12:01.792620 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:01.792558    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-observability-584f99c788-p45zp" podStartSLOduration=2.21216331 podStartE2EDuration="4.792538299s" totalImagesPullingTime="2.580374989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:01.765129907 +0000 UTC m=+353.506120986" watchObservedRunningTime="2026-10-30 19:12:01.792538299 +0000 UTC m=+353.533529381"
Oct 30 19:12:01.793255 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:01.793211    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk" podStartSLOduration=1.5299691690000001 podStartE2EDuration="4.793198396s" totalImagesPullingTime="3.263229227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:01.792151096 +0000 UTC m=+353.533142176" watchObservedRunningTime="2026-10-30 19:12:01.793198396 +0000 UTC m=+353.534189474"
Oct 30 19:12:02.753555 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:02.753519    4524 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-4fwqk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:12:02.753931 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:02.753574    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk" podUID="19679075-db01-441b-a98b-1378cfe9ddbc" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:12:05.752821 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:05.752791    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:12:12.754437 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:12.754405    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-observability-584f99c788-p45zp"
Oct 30 19:12:15.624160 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:12:15.624121    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:12:45.623645 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:12:45.623609    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:12:52.938275 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:52.938190    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-86vsp"]
Oct 30 19:12:52.941462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:52.941388    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-86vsp"
Oct 30 19:12:52.950278 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:52.950261    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-flsxp\""
Oct 30 19:12:52.950417 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:52.950284    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Oct 30 19:12:52.956619 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:52.956596    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-86vsp"]
Oct 30 19:12:53.037240 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.037213    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67bbe241-581a-468f-a88f-1b48afdfdc1d-cert\") pod \"kserve-controller-manager-776cc66d47-86vsp\" (UID: \"67bbe241-581a-468f-a88f-1b48afdfdc1d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-86vsp"
Oct 30 19:12:53.037385 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.037257    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scvjm\" (UniqueName: \"kubernetes.io/projected/67bbe241-581a-468f-a88f-1b48afdfdc1d-kube-api-access-scvjm\") pod \"kserve-controller-manager-776cc66d47-86vsp\" (UID: \"67bbe241-581a-468f-a88f-1b48afdfdc1d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-86vsp"
Oct 30 19:12:53.053139 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.053109    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-f87fcf56c-kxjpj"]
Oct 30 19:12:53.055312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.055296    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-f87fcf56c-kxjpj"
Oct 30 19:12:53.066828 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.066810    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Oct 30 19:12:53.066994 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.066983    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-nvbl8\""
Oct 30 19:12:53.067060 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.067045    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Oct 30 19:12:53.073340 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.073320    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-f87fcf56c-kxjpj"]
Oct 30 19:12:53.137981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.137953    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scvjm\" (UniqueName: \"kubernetes.io/projected/67bbe241-581a-468f-a88f-1b48afdfdc1d-kube-api-access-scvjm\") pod \"kserve-controller-manager-776cc66d47-86vsp\" (UID: \"67bbe241-581a-468f-a88f-1b48afdfdc1d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-86vsp"
Oct 30 19:12:53.138114 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.137991    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljk2r\" (UniqueName: \"kubernetes.io/projected/fd887584-7460-4fb8-bb36-f826836156c1-kube-api-access-ljk2r\") pod \"odh-model-controller-f87fcf56c-kxjpj\" (UID: \"fd887584-7460-4fb8-bb36-f826836156c1\") " pod="opendatahub/odh-model-controller-f87fcf56c-kxjpj"
Oct 30 19:12:53.138114 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.138024    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd887584-7460-4fb8-bb36-f826836156c1-cert\") pod \"odh-model-controller-f87fcf56c-kxjpj\" (UID: \"fd887584-7460-4fb8-bb36-f826836156c1\") " pod="opendatahub/odh-model-controller-f87fcf56c-kxjpj"
Oct 30 19:12:53.138114 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.138099    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd887584-7460-4fb8-bb36-f826836156c1-metrics-certs\") pod \"odh-model-controller-f87fcf56c-kxjpj\" (UID: \"fd887584-7460-4fb8-bb36-f826836156c1\") " pod="opendatahub/odh-model-controller-f87fcf56c-kxjpj"
Oct 30 19:12:53.138272 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.138205    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67bbe241-581a-468f-a88f-1b48afdfdc1d-cert\") pod \"kserve-controller-manager-776cc66d47-86vsp\" (UID: \"67bbe241-581a-468f-a88f-1b48afdfdc1d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-86vsp"
Oct 30 19:12:53.140734 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.140709    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67bbe241-581a-468f-a88f-1b48afdfdc1d-cert\") pod \"kserve-controller-manager-776cc66d47-86vsp\" (UID: \"67bbe241-581a-468f-a88f-1b48afdfdc1d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-86vsp"
Oct 30 19:12:53.150320 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.150301    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scvjm\" (UniqueName: \"kubernetes.io/projected/67bbe241-581a-468f-a88f-1b48afdfdc1d-kube-api-access-scvjm\") pod \"kserve-controller-manager-776cc66d47-86vsp\" (UID: \"67bbe241-581a-468f-a88f-1b48afdfdc1d\") " pod="opendatahub/kserve-controller-manager-776cc66d47-86vsp"
Oct 30 19:12:53.239564 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.239500    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljk2r\" (UniqueName: \"kubernetes.io/projected/fd887584-7460-4fb8-bb36-f826836156c1-kube-api-access-ljk2r\") pod \"odh-model-controller-f87fcf56c-kxjpj\" (UID: \"fd887584-7460-4fb8-bb36-f826836156c1\") " pod="opendatahub/odh-model-controller-f87fcf56c-kxjpj"
Oct 30 19:12:53.239564 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.239542    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd887584-7460-4fb8-bb36-f826836156c1-cert\") pod \"odh-model-controller-f87fcf56c-kxjpj\" (UID: \"fd887584-7460-4fb8-bb36-f826836156c1\") " pod="opendatahub/odh-model-controller-f87fcf56c-kxjpj"
Oct 30 19:12:53.239737 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.239574    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd887584-7460-4fb8-bb36-f826836156c1-metrics-certs\") pod \"odh-model-controller-f87fcf56c-kxjpj\" (UID: \"fd887584-7460-4fb8-bb36-f826836156c1\") " pod="opendatahub/odh-model-controller-f87fcf56c-kxjpj"
Oct 30 19:12:53.241810 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.241786    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd887584-7460-4fb8-bb36-f826836156c1-metrics-certs\") pod \"odh-model-controller-f87fcf56c-kxjpj\" (UID: \"fd887584-7460-4fb8-bb36-f826836156c1\") " pod="opendatahub/odh-model-controller-f87fcf56c-kxjpj"
Oct 30 19:12:53.241897 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.241829    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd887584-7460-4fb8-bb36-f826836156c1-cert\") pod \"odh-model-controller-f87fcf56c-kxjpj\" (UID: \"fd887584-7460-4fb8-bb36-f826836156c1\") " pod="opendatahub/odh-model-controller-f87fcf56c-kxjpj"
Oct 30 19:12:53.251630 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.251606    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljk2r\" (UniqueName: \"kubernetes.io/projected/fd887584-7460-4fb8-bb36-f826836156c1-kube-api-access-ljk2r\") pod \"odh-model-controller-f87fcf56c-kxjpj\" (UID: \"fd887584-7460-4fb8-bb36-f826836156c1\") " pod="opendatahub/odh-model-controller-f87fcf56c-kxjpj"
Oct 30 19:12:53.259460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.259441    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-86vsp"
Oct 30 19:12:53.302692 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.302661    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"]
Oct 30 19:12:53.305850 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.305479    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.322343 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.321809    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"]
Oct 30 19:12:53.367187 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.367145    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-f87fcf56c-kxjpj"
Oct 30 19:12:53.441472 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.441444    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/741a2f85-9f59-443e-909f-ff76c3816645-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.441589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.441476    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/741a2f85-9f59-443e-909f-ff76c3816645-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.441629 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.441603    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/741a2f85-9f59-443e-909f-ff76c3816645-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.441683 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.441669    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/741a2f85-9f59-443e-909f-ff76c3816645-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.441718 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.441705    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/741a2f85-9f59-443e-909f-ff76c3816645-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.441750 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.441724    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/741a2f85-9f59-443e-909f-ff76c3816645-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.441800 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.441780    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-996p5\" (UniqueName: \"kubernetes.io/projected/741a2f85-9f59-443e-909f-ff76c3816645-kube-api-access-996p5\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.536640 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.536601    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-86vsp"]
Oct 30 19:12:53.539237 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:12:53.539213    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67bbe241_581a_468f_a88f_1b48afdfdc1d.slice/crio-5db0780bf9e55d5e63e7adf877a36d7962a37cedc53987a0a729e43e344886c9 WatchSource:0}: Error finding container 5db0780bf9e55d5e63e7adf877a36d7962a37cedc53987a0a729e43e344886c9: Status 404 returned error can't find the container with id 5db0780bf9e55d5e63e7adf877a36d7962a37cedc53987a0a729e43e344886c9
Oct 30 19:12:53.542230 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.542208    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/741a2f85-9f59-443e-909f-ff76c3816645-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.542305 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.542237    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/741a2f85-9f59-443e-909f-ff76c3816645-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.542305 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.542254    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-996p5\" (UniqueName: \"kubernetes.io/projected/741a2f85-9f59-443e-909f-ff76c3816645-kube-api-access-996p5\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.542305 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.542283    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/741a2f85-9f59-443e-909f-ff76c3816645-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.542305 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.542300    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/741a2f85-9f59-443e-909f-ff76c3816645-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.542463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.542406    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/741a2f85-9f59-443e-909f-ff76c3816645-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.542513 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.542455    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/741a2f85-9f59-443e-909f-ff76c3816645-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.543128 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.543103    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/741a2f85-9f59-443e-909f-ff76c3816645-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.544684 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.544663    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/741a2f85-9f59-443e-909f-ff76c3816645-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.544780 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.544695    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/741a2f85-9f59-443e-909f-ff76c3816645-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.544854 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.544841    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/741a2f85-9f59-443e-909f-ff76c3816645-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.544940 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.544919    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/741a2f85-9f59-443e-909f-ff76c3816645-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.553074 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.553056    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/741a2f85-9f59-443e-909f-ff76c3816645-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.553526 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.553506    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-996p5\" (UniqueName: \"kubernetes.io/projected/741a2f85-9f59-443e-909f-ff76c3816645-kube-api-access-996p5\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2ltm4\" (UID: \"741a2f85-9f59-443e-909f-ff76c3816645\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.619344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.619323    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.654197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.654151    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-f87fcf56c-kxjpj"]
Oct 30 19:12:53.656340 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:12:53.656310    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd887584_7460_4fb8_bb36_f826836156c1.slice/crio-bfa3f2fbf346345ce3922320b0b0878820c67c067cb814f3fd02385c60b18b13 WatchSource:0}: Error finding container bfa3f2fbf346345ce3922320b0b0878820c67c067cb814f3fd02385c60b18b13: Status 404 returned error can't find the container with id bfa3f2fbf346345ce3922320b0b0878820c67c067cb814f3fd02385c60b18b13
Oct 30 19:12:53.895001 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.894972    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"]
Oct 30 19:12:53.897841 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:12:53.897817    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod741a2f85_9f59_443e_909f_ff76c3816645.slice/crio-14414ea871cf26443bf02e3c7ba1ecb7a0e0a61cb369e41e2e9171175ea25d42 WatchSource:0}: Error finding container 14414ea871cf26443bf02e3c7ba1ecb7a0e0a61cb369e41e2e9171175ea25d42: Status 404 returned error can't find the container with id 14414ea871cf26443bf02e3c7ba1ecb7a0e0a61cb369e41e2e9171175ea25d42
Oct 30 19:12:53.899901 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.899869    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:12:53.899980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.899931    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:12:53.965041 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.965008    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4" event={"ID":"741a2f85-9f59-443e-909f-ff76c3816645","Type":"ContainerStarted","Data":"977ef8bdc6691f8187f8f67fb78ed4bb7aa3a0ead9185c08d73a4568c6dbbe24"}
Oct 30 19:12:53.965439 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.965065    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4" event={"ID":"741a2f85-9f59-443e-909f-ff76c3816645","Type":"ContainerStarted","Data":"14414ea871cf26443bf02e3c7ba1ecb7a0e0a61cb369e41e2e9171175ea25d42"}
Oct 30 19:12:53.965439 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.965116    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:53.966511 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.966488    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-f87fcf56c-kxjpj" event={"ID":"fd887584-7460-4fb8-bb36-f826836156c1","Type":"ContainerStarted","Data":"bfa3f2fbf346345ce3922320b0b0878820c67c067cb814f3fd02385c60b18b13"}
Oct 30 19:12:53.967917 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.967891    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-86vsp" event={"ID":"67bbe241-581a-468f-a88f-1b48afdfdc1d","Type":"ContainerStarted","Data":"5db0780bf9e55d5e63e7adf877a36d7962a37cedc53987a0a729e43e344886c9"}
Oct 30 19:12:53.986908 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:53.986868    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4" podStartSLOduration=0.986855752 podStartE2EDuration="986.855752ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:53.986489183 +0000 UTC m=+405.727480263" watchObservedRunningTime="2026-10-30 19:12:53.986855752 +0000 UTC m=+405.727846832"
Oct 30 19:12:54.975795 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:54.975757    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2ltm4"
Oct 30 19:12:55.039823 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.039775    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"]
Oct 30 19:12:55.040039 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.039980    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk" podUID="19679075-db01-441b-a98b-1378cfe9ddbc" containerName="discovery" containerID="cri-o://56ab84ca70e827ae323c054cb65e718016ed5ce5eb63ec20dab6c64cbbb2917a" gracePeriod=30
Oct 30 19:12:55.343683 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.343653    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:12:55.462301 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.462261    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-istio-csr-dns-cert\") pod \"19679075-db01-441b-a98b-1378cfe9ddbc\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") "
Oct 30 19:12:55.462465 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.462330    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/19679075-db01-441b-a98b-1378cfe9ddbc-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/19679075-db01-441b-a98b-1378cfe9ddbc-local-certs\") pod \"19679075-db01-441b-a98b-1378cfe9ddbc\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") "
Oct 30 19:12:55.462465 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.462367    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-cacerts\" (UniqueName: \"kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-cacerts\") pod \"19679075-db01-441b-a98b-1378cfe9ddbc\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") "
Oct 30 19:12:55.462465 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.462405    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/19679075-db01-441b-a98b-1378cfe9ddbc-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/19679075-db01-441b-a98b-1378cfe9ddbc-istio-csr-ca-configmap\") pod \"19679075-db01-441b-a98b-1378cfe9ddbc\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") "
Oct 30 19:12:55.462622 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.462550    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19679075-db01-441b-a98b-1378cfe9ddbc-kube-api-access-qhqrn\" (UniqueName: \"kubernetes.io/projected/19679075-db01-441b-a98b-1378cfe9ddbc-kube-api-access-qhqrn\") pod \"19679075-db01-441b-a98b-1378cfe9ddbc\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") "
Oct 30 19:12:55.462669 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.462620    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-istio-kubeconfig\") pod \"19679075-db01-441b-a98b-1378cfe9ddbc\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") "
Oct 30 19:12:55.462719 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.462678    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/19679075-db01-441b-a98b-1378cfe9ddbc-istio-token\" (UniqueName: \"kubernetes.io/projected/19679075-db01-441b-a98b-1378cfe9ddbc-istio-token\") pod \"19679075-db01-441b-a98b-1378cfe9ddbc\" (UID: \"19679075-db01-441b-a98b-1378cfe9ddbc\") "
Oct 30 19:12:55.463440 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.463387    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19679075-db01-441b-a98b-1378cfe9ddbc-istio-csr-ca-configmap" pod "19679075-db01-441b-a98b-1378cfe9ddbc" (UID: "19679075-db01-441b-a98b-1378cfe9ddbc"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:12:55.469972 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.469848    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19679075-db01-441b-a98b-1378cfe9ddbc-kube-api-access-qhqrn" pod "19679075-db01-441b-a98b-1378cfe9ddbc" (UID: "19679075-db01-441b-a98b-1378cfe9ddbc"). InnerVolumeSpecName "kube-api-access-qhqrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:12:55.470345 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.470229    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19679075-db01-441b-a98b-1378cfe9ddbc-local-certs" pod "19679075-db01-441b-a98b-1378cfe9ddbc" (UID: "19679075-db01-441b-a98b-1378cfe9ddbc"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:12:55.470345 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.470242    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19679075-db01-441b-a98b-1378cfe9ddbc-istio-token" pod "19679075-db01-441b-a98b-1378cfe9ddbc" (UID: "19679075-db01-441b-a98b-1378cfe9ddbc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:12:55.470345 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.470248    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-istio-kubeconfig" pod "19679075-db01-441b-a98b-1378cfe9ddbc" (UID: "19679075-db01-441b-a98b-1378cfe9ddbc"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:12:55.470345 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.470300    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-cacerts" pod "19679075-db01-441b-a98b-1378cfe9ddbc" (UID: "19679075-db01-441b-a98b-1378cfe9ddbc"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:12:55.470609 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.470386    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-istio-csr-dns-cert" pod "19679075-db01-441b-a98b-1378cfe9ddbc" (UID: "19679075-db01-441b-a98b-1378cfe9ddbc"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:12:55.564072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.563990    4524 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-istio-csr-dns-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:55.564072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.564023    4524 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/19679075-db01-441b-a98b-1378cfe9ddbc-local-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:55.564072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.564037    4524 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-cacerts\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:55.564072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.564047    4524 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/19679075-db01-441b-a98b-1378cfe9ddbc-istio-csr-ca-configmap\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:55.564072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.564056    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhqrn\" (UniqueName: \"kubernetes.io/projected/19679075-db01-441b-a98b-1378cfe9ddbc-kube-api-access-qhqrn\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:55.564072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.564068    4524 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/19679075-db01-441b-a98b-1378cfe9ddbc-istio-kubeconfig\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:55.564525 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.564082    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19679075-db01-441b-a98b-1378cfe9ddbc-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:12:55.981478 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.981442    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-86vsp" event={"ID":"67bbe241-581a-468f-a88f-1b48afdfdc1d","Type":"ContainerStarted","Data":"1d9b2adf8442b2c0c8163e449d6344a7b476f314ee7e3ea5533e3fbfbd81db96"}
Oct 30 19:12:55.981892 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.981549    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-86vsp"
Oct 30 19:12:55.982930 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.982904    4524 generic.go:356] "Generic (PLEG): container finished" podID="19679075-db01-441b-a98b-1378cfe9ddbc" containerID="56ab84ca70e827ae323c054cb65e718016ed5ce5eb63ec20dab6c64cbbb2917a" exitCode=0
Oct 30 19:12:55.983055 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.982975    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"
Oct 30 19:12:55.983055 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.983019    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk" event={"ID":"19679075-db01-441b-a98b-1378cfe9ddbc","Type":"ContainerDied","Data":"56ab84ca70e827ae323c054cb65e718016ed5ce5eb63ec20dab6c64cbbb2917a"}
Oct 30 19:12:55.983186 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.983063    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk" event={"ID":"19679075-db01-441b-a98b-1378cfe9ddbc","Type":"ContainerDied","Data":"a00767e439850575623cb54b43a44b20be00fe6305c3e95833da35ee482dd3d5"}
Oct 30 19:12:55.983186 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.983085    4524 scope.go:122] "RemoveContainer" containerID="56ab84ca70e827ae323c054cb65e718016ed5ce5eb63ec20dab6c64cbbb2917a"
Oct 30 19:12:55.997384 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.997358    4524 scope.go:122] "RemoveContainer" containerID="56ab84ca70e827ae323c054cb65e718016ed5ce5eb63ec20dab6c64cbbb2917a"
Oct 30 19:12:55.997646 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:12:55.997625    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56ab84ca70e827ae323c054cb65e718016ed5ce5eb63ec20dab6c64cbbb2917a\": container with ID starting with 56ab84ca70e827ae323c054cb65e718016ed5ce5eb63ec20dab6c64cbbb2917a not found: ID does not exist" containerID="56ab84ca70e827ae323c054cb65e718016ed5ce5eb63ec20dab6c64cbbb2917a"
Oct 30 19:12:55.997731 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:55.997653    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56ab84ca70e827ae323c054cb65e718016ed5ce5eb63ec20dab6c64cbbb2917a"} err="failed to get container status \"56ab84ca70e827ae323c054cb65e718016ed5ce5eb63ec20dab6c64cbbb2917a\": rpc error: code = NotFound desc = could not find container \"56ab84ca70e827ae323c054cb65e718016ed5ce5eb63ec20dab6c64cbbb2917a\": container with ID starting with 56ab84ca70e827ae323c054cb65e718016ed5ce5eb63ec20dab6c64cbbb2917a not found: ID does not exist"
Oct 30 19:12:56.002451 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:56.002410    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-86vsp" podStartSLOduration=1.657726493 podStartE2EDuration="4.002393819s" totalImagesPullingTime="2.344667326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:56.00067626 +0000 UTC m=+407.741667363" watchObservedRunningTime="2026-10-30 19:12:56.002393819 +0000 UTC m=+407.743384901"
Oct 30 19:12:56.015612 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:56.015589    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"]
Oct 30 19:12:56.019618 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:56.019588    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-4fwqk"]
Oct 30 19:12:56.390133 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:56.390092    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="19679075-db01-441b-a98b-1378cfe9ddbc" path="/var/lib/kubelet/pods/19679075-db01-441b-a98b-1378cfe9ddbc/volumes"
Oct 30 19:12:56.989435 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:56.989401    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-f87fcf56c-kxjpj" event={"ID":"fd887584-7460-4fb8-bb36-f826836156c1","Type":"ContainerStarted","Data":"04cff2e03d10e9329b3f41e08ecf58aba882cc6c6997b08457bc6cd742b30d0b"}
Oct 30 19:12:57.007818 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:57.007756    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-f87fcf56c-kxjpj" podStartSLOduration=1.161591919 podStartE2EDuration="4.007742461s" totalImagesPullingTime="2.846150542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:57.006436571 +0000 UTC m=+408.747427650" watchObservedRunningTime="2026-10-30 19:12:57.007742461 +0000 UTC m=+408.748733539"
Oct 30 19:12:57.994497 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:12:57.994458    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-f87fcf56c-kxjpj"
Oct 30 19:13:09.001608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:09.001578    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-f87fcf56c-kxjpj"
Oct 30 19:13:15.623701 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:13:15.623666    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:13:26.994318 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:26.994289    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-86vsp"
Oct 30 19:13:45.624808 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:13:45.624768    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:13:52.258286 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.258255    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs"]
Oct 30 19:13:52.258657 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.258565    4524 cpu_manager.go:414] "containerMap: removing container" podUID="19679075-db01-441b-a98b-1378cfe9ddbc" containerName="discovery"
Oct 30 19:13:52.258657 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.258575    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="19679075-db01-441b-a98b-1378cfe9ddbc" containerName="discovery"
Oct 30 19:13:52.258657 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.258628    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="19679075-db01-441b-a98b-1378cfe9ddbc" containerName="discovery"
Oct 30 19:13:52.260691 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.260671    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs"
Oct 30 19:13:52.264539 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.264517    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 19:13:52.264920 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.264902    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-5pdhx\""
Oct 30 19:13:52.264978 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.264929    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 19:13:52.271834 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.271813    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs"]
Oct 30 19:13:52.291881 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.291859    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6340301b-511c-4c80-bae8-1bd4582b2d65-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs\" (UID: \"6340301b-511c-4c80-bae8-1bd4582b2d65\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs"
Oct 30 19:13:52.291979 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.291951    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8klt\" (UniqueName: \"kubernetes.io/projected/6340301b-511c-4c80-bae8-1bd4582b2d65-kube-api-access-k8klt\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs\" (UID: \"6340301b-511c-4c80-bae8-1bd4582b2d65\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs"
Oct 30 19:13:52.292113 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.292095    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6340301b-511c-4c80-bae8-1bd4582b2d65-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs\" (UID: \"6340301b-511c-4c80-bae8-1bd4582b2d65\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs"
Oct 30 19:13:52.354330 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.354308    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv"]
Oct 30 19:13:52.356910 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.356894    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv"
Oct 30 19:13:52.365720 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.365699    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-v2pks\""
Oct 30 19:13:52.368759 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.368725    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv"]
Oct 30 19:13:52.393408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.393389    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1125a460-6a15-4179-a5fa-00aa4fc10968-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv\" (UID: \"1125a460-6a15-4179-a5fa-00aa4fc10968\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv"
Oct 30 19:13:52.393508 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.393428    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1125a460-6a15-4179-a5fa-00aa4fc10968-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv\" (UID: \"1125a460-6a15-4179-a5fa-00aa4fc10968\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv"
Oct 30 19:13:52.393508 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.393451    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6340301b-511c-4c80-bae8-1bd4582b2d65-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs\" (UID: \"6340301b-511c-4c80-bae8-1bd4582b2d65\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs"
Oct 30 19:13:52.393635 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.393616    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6340301b-511c-4c80-bae8-1bd4582b2d65-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs\" (UID: \"6340301b-511c-4c80-bae8-1bd4582b2d65\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs"
Oct 30 19:13:52.393673 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.393646    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psch8\" (UniqueName: \"kubernetes.io/projected/1125a460-6a15-4179-a5fa-00aa4fc10968-kube-api-access-psch8\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv\" (UID: \"1125a460-6a15-4179-a5fa-00aa4fc10968\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv"
Oct 30 19:13:52.393708 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.393676    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8klt\" (UniqueName: \"kubernetes.io/projected/6340301b-511c-4c80-bae8-1bd4582b2d65-kube-api-access-k8klt\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs\" (UID: \"6340301b-511c-4c80-bae8-1bd4582b2d65\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs"
Oct 30 19:13:52.393759 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.393743    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6340301b-511c-4c80-bae8-1bd4582b2d65-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs\" (UID: \"6340301b-511c-4c80-bae8-1bd4582b2d65\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs"
Oct 30 19:13:52.394013 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.393994    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6340301b-511c-4c80-bae8-1bd4582b2d65-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs\" (UID: \"6340301b-511c-4c80-bae8-1bd4582b2d65\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs"
Oct 30 19:13:52.405784 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.405760    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8klt\" (UniqueName: \"kubernetes.io/projected/6340301b-511c-4c80-bae8-1bd4582b2d65-kube-api-access-k8klt\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs\" (UID: \"6340301b-511c-4c80-bae8-1bd4582b2d65\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs"
Oct 30 19:13:52.457938 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.457913    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv"]
Oct 30 19:13:52.460438 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.460422    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv"
Oct 30 19:13:52.466004 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.465985    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-pzmvk\""
Oct 30 19:13:52.471448 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.471426    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv"]
Oct 30 19:13:52.494165 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.494139    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg94z\" (UniqueName: \"kubernetes.io/projected/97e76769-ac02-408e-8c04-10bb5c7ff179-kube-api-access-dg94z\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv\" (UID: \"97e76769-ac02-408e-8c04-10bb5c7ff179\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv"
Oct 30 19:13:52.494293 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.494216    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1125a460-6a15-4179-a5fa-00aa4fc10968-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv\" (UID: \"1125a460-6a15-4179-a5fa-00aa4fc10968\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv"
Oct 30 19:13:52.494365 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.494314    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1125a460-6a15-4179-a5fa-00aa4fc10968-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv\" (UID: \"1125a460-6a15-4179-a5fa-00aa4fc10968\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv"
Oct 30 19:13:52.494365 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.494354    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97e76769-ac02-408e-8c04-10bb5c7ff179-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv\" (UID: \"97e76769-ac02-408e-8c04-10bb5c7ff179\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv"
Oct 30 19:13:52.494471 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.494391    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97e76769-ac02-408e-8c04-10bb5c7ff179-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv\" (UID: \"97e76769-ac02-408e-8c04-10bb5c7ff179\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv"
Oct 30 19:13:52.494471 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.494431    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psch8\" (UniqueName: \"kubernetes.io/projected/1125a460-6a15-4179-a5fa-00aa4fc10968-kube-api-access-psch8\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv\" (UID: \"1125a460-6a15-4179-a5fa-00aa4fc10968\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv"
Oct 30 19:13:52.494581 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.494561    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1125a460-6a15-4179-a5fa-00aa4fc10968-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv\" (UID: \"1125a460-6a15-4179-a5fa-00aa4fc10968\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv"
Oct 30 19:13:52.494626 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.494592    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1125a460-6a15-4179-a5fa-00aa4fc10968-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv\" (UID: \"1125a460-6a15-4179-a5fa-00aa4fc10968\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv"
Oct 30 19:13:52.506144 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.506121    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psch8\" (UniqueName: \"kubernetes.io/projected/1125a460-6a15-4179-a5fa-00aa4fc10968-kube-api-access-psch8\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv\" (UID: \"1125a460-6a15-4179-a5fa-00aa4fc10968\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv"
Oct 30 19:13:52.558522 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.558474    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8"]
Oct 30 19:13:52.560867 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.560853    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8"
Oct 30 19:13:52.569448 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.569433    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-6nmfg\""
Oct 30 19:13:52.575181 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.575142    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs"
Oct 30 19:13:52.578198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.577656    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8"]
Oct 30 19:13:52.595781 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.595747    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvsgx\" (UniqueName: \"kubernetes.io/projected/7748a000-8451-4c97-8e69-3fd63eafc781-kube-api-access-cvsgx\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8\" (UID: \"7748a000-8451-4c97-8e69-3fd63eafc781\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8"
Oct 30 19:13:52.595882 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.595816    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg94z\" (UniqueName: \"kubernetes.io/projected/97e76769-ac02-408e-8c04-10bb5c7ff179-kube-api-access-dg94z\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv\" (UID: \"97e76769-ac02-408e-8c04-10bb5c7ff179\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv"
Oct 30 19:13:52.595973 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.595939    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7748a000-8451-4c97-8e69-3fd63eafc781-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8\" (UID: \"7748a000-8451-4c97-8e69-3fd63eafc781\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8"
Oct 30 19:13:52.596072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.596050    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7748a000-8451-4c97-8e69-3fd63eafc781-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8\" (UID: \"7748a000-8451-4c97-8e69-3fd63eafc781\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8"
Oct 30 19:13:52.596155 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.596137    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97e76769-ac02-408e-8c04-10bb5c7ff179-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv\" (UID: \"97e76769-ac02-408e-8c04-10bb5c7ff179\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv"
Oct 30 19:13:52.596255 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.596230    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97e76769-ac02-408e-8c04-10bb5c7ff179-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv\" (UID: \"97e76769-ac02-408e-8c04-10bb5c7ff179\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv"
Oct 30 19:13:52.596611 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.596589    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97e76769-ac02-408e-8c04-10bb5c7ff179-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv\" (UID: \"97e76769-ac02-408e-8c04-10bb5c7ff179\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv"
Oct 30 19:13:52.596676 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.596592    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97e76769-ac02-408e-8c04-10bb5c7ff179-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv\" (UID: \"97e76769-ac02-408e-8c04-10bb5c7ff179\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv"
Oct 30 19:13:52.607679 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.607650    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg94z\" (UniqueName: \"kubernetes.io/projected/97e76769-ac02-408e-8c04-10bb5c7ff179-kube-api-access-dg94z\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv\" (UID: \"97e76769-ac02-408e-8c04-10bb5c7ff179\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv"
Oct 30 19:13:52.670744 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.670718    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv"
Oct 30 19:13:52.697516 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.697486    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvsgx\" (UniqueName: \"kubernetes.io/projected/7748a000-8451-4c97-8e69-3fd63eafc781-kube-api-access-cvsgx\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8\" (UID: \"7748a000-8451-4c97-8e69-3fd63eafc781\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8"
Oct 30 19:13:52.698107 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.697557    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7748a000-8451-4c97-8e69-3fd63eafc781-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8\" (UID: \"7748a000-8451-4c97-8e69-3fd63eafc781\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8"
Oct 30 19:13:52.698107 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.697588    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7748a000-8451-4c97-8e69-3fd63eafc781-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8\" (UID: \"7748a000-8451-4c97-8e69-3fd63eafc781\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8"
Oct 30 19:13:52.698107 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.697960    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7748a000-8451-4c97-8e69-3fd63eafc781-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8\" (UID: \"7748a000-8451-4c97-8e69-3fd63eafc781\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8"
Oct 30 19:13:52.698107 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.697975    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7748a000-8451-4c97-8e69-3fd63eafc781-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8\" (UID: \"7748a000-8451-4c97-8e69-3fd63eafc781\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8"
Oct 30 19:13:52.709470 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.709447    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvsgx\" (UniqueName: \"kubernetes.io/projected/7748a000-8451-4c97-8e69-3fd63eafc781-kube-api-access-cvsgx\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8\" (UID: \"7748a000-8451-4c97-8e69-3fd63eafc781\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8"
Oct 30 19:13:52.773446 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.773416    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv"
Oct 30 19:13:52.850650 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.850628    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs"]
Oct 30 19:13:52.852529 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:13:52.852507    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6340301b_511c_4c80_bae8_1bd4582b2d65.slice/crio-806a19ec66d7a5508a214138e28cb1f89bb1a6a88944f2822c92f4f11b44455a WatchSource:0}: Error finding container 806a19ec66d7a5508a214138e28cb1f89bb1a6a88944f2822c92f4f11b44455a: Status 404 returned error can't find the container with id 806a19ec66d7a5508a214138e28cb1f89bb1a6a88944f2822c92f4f11b44455a
Oct 30 19:13:52.874190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.874149    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8"
Oct 30 19:13:52.946561 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:52.946537    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv"]
Oct 30 19:13:53.006945 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:13:53.006913    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1125a460_6a15_4179_a5fa_00aa4fc10968.slice/crio-e385a51f5db390569804686c2c41304d7dac8f01dd3e3f5d2982a574701f9d55 WatchSource:0}: Error finding container e385a51f5db390569804686c2c41304d7dac8f01dd3e3f5d2982a574701f9d55: Status 404 returned error can't find the container with id e385a51f5db390569804686c2c41304d7dac8f01dd3e3f5d2982a574701f9d55
Oct 30 19:13:53.044482 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:53.044433    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv"]
Oct 30 19:13:53.046565 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:13:53.046539    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97e76769_ac02_408e_8c04_10bb5c7ff179.slice/crio-d7ef3a7b4c14e4309985c082f7e61ee4e1fbec16f71b8f4095eb647cf8e19dcf WatchSource:0}: Error finding container d7ef3a7b4c14e4309985c082f7e61ee4e1fbec16f71b8f4095eb647cf8e19dcf: Status 404 returned error can't find the container with id d7ef3a7b4c14e4309985c082f7e61ee4e1fbec16f71b8f4095eb647cf8e19dcf
Oct 30 19:13:53.146805 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:53.146774    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8"]
Oct 30 19:13:53.200895 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:13:53.200869    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7748a000_8451_4c97_8e69_3fd63eafc781.slice/crio-7dc15c4ca976c6057b5b19ffc72de7af2a53afd8d7987cd24f5019e51cf148b7 WatchSource:0}: Error finding container 7dc15c4ca976c6057b5b19ffc72de7af2a53afd8d7987cd24f5019e51cf148b7: Status 404 returned error can't find the container with id 7dc15c4ca976c6057b5b19ffc72de7af2a53afd8d7987cd24f5019e51cf148b7
Oct 30 19:13:53.228434 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:53.228408    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8" event={"ID":"7748a000-8451-4c97-8e69-3fd63eafc781","Type":"ContainerStarted","Data":"7dc15c4ca976c6057b5b19ffc72de7af2a53afd8d7987cd24f5019e51cf148b7"}
Oct 30 19:13:53.229754 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:53.229731    4524 generic.go:356] "Generic (PLEG): container finished" podID="1125a460-6a15-4179-a5fa-00aa4fc10968" containerID="9d107b823aa4294fee5d3b54840410247751ff876ccd671acfa4ccabaad15551" exitCode=0
Oct 30 19:13:53.229852 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:53.229811    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv" event={"ID":"1125a460-6a15-4179-a5fa-00aa4fc10968","Type":"ContainerDied","Data":"9d107b823aa4294fee5d3b54840410247751ff876ccd671acfa4ccabaad15551"}
Oct 30 19:13:53.229910 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:53.229866    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv" event={"ID":"1125a460-6a15-4179-a5fa-00aa4fc10968","Type":"ContainerStarted","Data":"e385a51f5db390569804686c2c41304d7dac8f01dd3e3f5d2982a574701f9d55"}
Oct 30 19:13:53.231238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:53.231216    4524 generic.go:356] "Generic (PLEG): container finished" podID="6340301b-511c-4c80-bae8-1bd4582b2d65" containerID="9695dd1befaa3729c801931178ea708aed989d634cd4ab6322e33593aff49441" exitCode=0
Oct 30 19:13:53.231318 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:53.231294    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs" event={"ID":"6340301b-511c-4c80-bae8-1bd4582b2d65","Type":"ContainerDied","Data":"9695dd1befaa3729c801931178ea708aed989d634cd4ab6322e33593aff49441"}
Oct 30 19:13:53.231378 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:53.231332    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs" event={"ID":"6340301b-511c-4c80-bae8-1bd4582b2d65","Type":"ContainerStarted","Data":"806a19ec66d7a5508a214138e28cb1f89bb1a6a88944f2822c92f4f11b44455a"}
Oct 30 19:13:53.233155 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:53.233138    4524 generic.go:356] "Generic (PLEG): container finished" podID="97e76769-ac02-408e-8c04-10bb5c7ff179" containerID="5e18e2bd0a01779ac47760f44e75191b2c7673a1ea1e7d6605a46f3bb1751aa9" exitCode=0
Oct 30 19:13:53.233222 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:53.233190    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv" event={"ID":"97e76769-ac02-408e-8c04-10bb5c7ff179","Type":"ContainerDied","Data":"5e18e2bd0a01779ac47760f44e75191b2c7673a1ea1e7d6605a46f3bb1751aa9"}
Oct 30 19:13:53.233271 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:53.233220    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv" event={"ID":"97e76769-ac02-408e-8c04-10bb5c7ff179","Type":"ContainerStarted","Data":"d7ef3a7b4c14e4309985c082f7e61ee4e1fbec16f71b8f4095eb647cf8e19dcf"}
Oct 30 19:13:54.240246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:54.240217    4524 generic.go:356] "Generic (PLEG): container finished" podID="7748a000-8451-4c97-8e69-3fd63eafc781" containerID="a33cd4c1e4ccc104eae5e5159a8ccaef046d74fc811c5ddb531975d51ca61956" exitCode=0
Oct 30 19:13:54.240556 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:54.240297    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8" event={"ID":"7748a000-8451-4c97-8e69-3fd63eafc781","Type":"ContainerDied","Data":"a33cd4c1e4ccc104eae5e5159a8ccaef046d74fc811c5ddb531975d51ca61956"}
Oct 30 19:13:55.246856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:55.246826    4524 generic.go:356] "Generic (PLEG): container finished" podID="97e76769-ac02-408e-8c04-10bb5c7ff179" containerID="0a9e81b45515d5dc6d4d13ce81e0821eabfe326445e75e520d92010f80d1bd27" exitCode=0
Oct 30 19:13:55.247299 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:55.246908    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv" event={"ID":"97e76769-ac02-408e-8c04-10bb5c7ff179","Type":"ContainerDied","Data":"0a9e81b45515d5dc6d4d13ce81e0821eabfe326445e75e520d92010f80d1bd27"}
Oct 30 19:13:55.248859 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:55.248830    4524 generic.go:356] "Generic (PLEG): container finished" podID="6340301b-511c-4c80-bae8-1bd4582b2d65" containerID="7bc42ff71350804701f1263863dee9070565c88b3eaf2338663f141f51b9d3c6" exitCode=0
Oct 30 19:13:55.248956 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:55.248880    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs" event={"ID":"6340301b-511c-4c80-bae8-1bd4582b2d65","Type":"ContainerDied","Data":"7bc42ff71350804701f1263863dee9070565c88b3eaf2338663f141f51b9d3c6"}
Oct 30 19:13:56.256247 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:56.256211    4524 generic.go:356] "Generic (PLEG): container finished" podID="97e76769-ac02-408e-8c04-10bb5c7ff179" containerID="84df50577ef3aa6c26ce9d4714256bc0f2fe284d6ba6a039f022fe1082438de7" exitCode=0
Oct 30 19:13:56.256640 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:56.256296    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv" event={"ID":"97e76769-ac02-408e-8c04-10bb5c7ff179","Type":"ContainerDied","Data":"84df50577ef3aa6c26ce9d4714256bc0f2fe284d6ba6a039f022fe1082438de7"}
Oct 30 19:13:56.257983 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:56.257957    4524 generic.go:356] "Generic (PLEG): container finished" podID="7748a000-8451-4c97-8e69-3fd63eafc781" containerID="c90c31ec2656168b169f4f5b39d710179c6e57a2f73de3db71c86d7d446f2f40" exitCode=0
Oct 30 19:13:56.258082 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:56.258037    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8" event={"ID":"7748a000-8451-4c97-8e69-3fd63eafc781","Type":"ContainerDied","Data":"c90c31ec2656168b169f4f5b39d710179c6e57a2f73de3db71c86d7d446f2f40"}
Oct 30 19:13:56.260541 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:56.260517    4524 generic.go:356] "Generic (PLEG): container finished" podID="6340301b-511c-4c80-bae8-1bd4582b2d65" containerID="f9995f3ae3178e0b935623adabae3e1581d824349e3ea0ab4d31a84c6e854b23" exitCode=0
Oct 30 19:13:56.260636 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:56.260552    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs" event={"ID":"6340301b-511c-4c80-bae8-1bd4582b2d65","Type":"ContainerDied","Data":"f9995f3ae3178e0b935623adabae3e1581d824349e3ea0ab4d31a84c6e854b23"}
Oct 30 19:13:57.266953 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.266918    4524 generic.go:356] "Generic (PLEG): container finished" podID="7748a000-8451-4c97-8e69-3fd63eafc781" containerID="67412061dc90190a49418b3af5e23ac88b22837a64cd958c112c0b5640f47246" exitCode=0
Oct 30 19:13:57.267405 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.266988    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8" event={"ID":"7748a000-8451-4c97-8e69-3fd63eafc781","Type":"ContainerDied","Data":"67412061dc90190a49418b3af5e23ac88b22837a64cd958c112c0b5640f47246"}
Oct 30 19:13:57.416756 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.416734    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs"
Oct 30 19:13:57.419462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.419445    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv"
Oct 30 19:13:57.532004 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.531943    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97e76769-ac02-408e-8c04-10bb5c7ff179-kube-api-access-dg94z\" (UniqueName: \"kubernetes.io/projected/97e76769-ac02-408e-8c04-10bb5c7ff179-kube-api-access-dg94z\") pod \"97e76769-ac02-408e-8c04-10bb5c7ff179\" (UID: \"97e76769-ac02-408e-8c04-10bb5c7ff179\") "
Oct 30 19:13:57.532004 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.531998    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6340301b-511c-4c80-bae8-1bd4582b2d65-util\" (UniqueName: \"kubernetes.io/empty-dir/6340301b-511c-4c80-bae8-1bd4582b2d65-util\") pod \"6340301b-511c-4c80-bae8-1bd4582b2d65\" (UID: \"6340301b-511c-4c80-bae8-1bd4582b2d65\") "
Oct 30 19:13:57.532162 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.532033    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6340301b-511c-4c80-bae8-1bd4582b2d65-kube-api-access-k8klt\" (UniqueName: \"kubernetes.io/projected/6340301b-511c-4c80-bae8-1bd4582b2d65-kube-api-access-k8klt\") pod \"6340301b-511c-4c80-bae8-1bd4582b2d65\" (UID: \"6340301b-511c-4c80-bae8-1bd4582b2d65\") "
Oct 30 19:13:57.532162 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.532049    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97e76769-ac02-408e-8c04-10bb5c7ff179-util\" (UniqueName: \"kubernetes.io/empty-dir/97e76769-ac02-408e-8c04-10bb5c7ff179-util\") pod \"97e76769-ac02-408e-8c04-10bb5c7ff179\" (UID: \"97e76769-ac02-408e-8c04-10bb5c7ff179\") "
Oct 30 19:13:57.532162 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.532106    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6340301b-511c-4c80-bae8-1bd4582b2d65-bundle\" (UniqueName: \"kubernetes.io/empty-dir/6340301b-511c-4c80-bae8-1bd4582b2d65-bundle\") pod \"6340301b-511c-4c80-bae8-1bd4582b2d65\" (UID: \"6340301b-511c-4c80-bae8-1bd4582b2d65\") "
Oct 30 19:13:57.532162 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.532137    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/97e76769-ac02-408e-8c04-10bb5c7ff179-bundle\" (UniqueName: \"kubernetes.io/empty-dir/97e76769-ac02-408e-8c04-10bb5c7ff179-bundle\") pod \"97e76769-ac02-408e-8c04-10bb5c7ff179\" (UID: \"97e76769-ac02-408e-8c04-10bb5c7ff179\") "
Oct 30 19:13:57.532803 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.532655    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6340301b-511c-4c80-bae8-1bd4582b2d65-bundle" pod "6340301b-511c-4c80-bae8-1bd4582b2d65" (UID: "6340301b-511c-4c80-bae8-1bd4582b2d65"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:57.532945 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.532919    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97e76769-ac02-408e-8c04-10bb5c7ff179-bundle" pod "97e76769-ac02-408e-8c04-10bb5c7ff179" (UID: "97e76769-ac02-408e-8c04-10bb5c7ff179"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:57.534117 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.534100    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6340301b-511c-4c80-bae8-1bd4582b2d65-kube-api-access-k8klt" pod "6340301b-511c-4c80-bae8-1bd4582b2d65" (UID: "6340301b-511c-4c80-bae8-1bd4582b2d65"). InnerVolumeSpecName "kube-api-access-k8klt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:57.534436 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.534422    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97e76769-ac02-408e-8c04-10bb5c7ff179-kube-api-access-dg94z" pod "97e76769-ac02-408e-8c04-10bb5c7ff179" (UID: "97e76769-ac02-408e-8c04-10bb5c7ff179"). InnerVolumeSpecName "kube-api-access-dg94z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:57.537561 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.537542    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97e76769-ac02-408e-8c04-10bb5c7ff179-util" pod "97e76769-ac02-408e-8c04-10bb5c7ff179" (UID: "97e76769-ac02-408e-8c04-10bb5c7ff179"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:57.540348 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.540320    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6340301b-511c-4c80-bae8-1bd4582b2d65-util" pod "6340301b-511c-4c80-bae8-1bd4582b2d65" (UID: "6340301b-511c-4c80-bae8-1bd4582b2d65"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:57.633149 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.633124    4524 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6340301b-511c-4c80-bae8-1bd4582b2d65-util\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:57.633149 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.633147    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8klt\" (UniqueName: \"kubernetes.io/projected/6340301b-511c-4c80-bae8-1bd4582b2d65-kube-api-access-k8klt\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:57.633291 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.633162    4524 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/97e76769-ac02-408e-8c04-10bb5c7ff179-util\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:57.633291 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.633195    4524 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6340301b-511c-4c80-bae8-1bd4582b2d65-bundle\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:57.633291 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.633207    4524 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/97e76769-ac02-408e-8c04-10bb5c7ff179-bundle\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:57.633291 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:57.633220    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dg94z\" (UniqueName: \"kubernetes.io/projected/97e76769-ac02-408e-8c04-10bb5c7ff179-kube-api-access-dg94z\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:58.273240 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:58.273211    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs"
Oct 30 19:13:58.273797 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:58.273211    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7jjrhs" event={"ID":"6340301b-511c-4c80-bae8-1bd4582b2d65","Type":"ContainerDied","Data":"806a19ec66d7a5508a214138e28cb1f89bb1a6a88944f2822c92f4f11b44455a"}
Oct 30 19:13:58.273797 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:58.273342    4524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="806a19ec66d7a5508a214138e28cb1f89bb1a6a88944f2822c92f4f11b44455a"
Oct 30 19:13:58.275324 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:58.275291    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv" event={"ID":"97e76769-ac02-408e-8c04-10bb5c7ff179","Type":"ContainerDied","Data":"d7ef3a7b4c14e4309985c082f7e61ee4e1fbec16f71b8f4095eb647cf8e19dcf"}
Oct 30 19:13:58.275449 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:58.275328    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ab2zpv"
Oct 30 19:13:58.275449 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:58.275355    4524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d7ef3a7b4c14e4309985c082f7e61ee4e1fbec16f71b8f4095eb647cf8e19dcf"
Oct 30 19:13:58.389254 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:58.389229    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8"
Oct 30 19:13:58.439557 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:58.439528    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7748a000-8451-4c97-8e69-3fd63eafc781-util\" (UniqueName: \"kubernetes.io/empty-dir/7748a000-8451-4c97-8e69-3fd63eafc781-util\") pod \"7748a000-8451-4c97-8e69-3fd63eafc781\" (UID: \"7748a000-8451-4c97-8e69-3fd63eafc781\") "
Oct 30 19:13:58.439691 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:58.439582    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7748a000-8451-4c97-8e69-3fd63eafc781-bundle\" (UniqueName: \"kubernetes.io/empty-dir/7748a000-8451-4c97-8e69-3fd63eafc781-bundle\") pod \"7748a000-8451-4c97-8e69-3fd63eafc781\" (UID: \"7748a000-8451-4c97-8e69-3fd63eafc781\") "
Oct 30 19:13:58.439691 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:58.439614    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7748a000-8451-4c97-8e69-3fd63eafc781-kube-api-access-cvsgx\" (UniqueName: \"kubernetes.io/projected/7748a000-8451-4c97-8e69-3fd63eafc781-kube-api-access-cvsgx\") pod \"7748a000-8451-4c97-8e69-3fd63eafc781\" (UID: \"7748a000-8451-4c97-8e69-3fd63eafc781\") "
Oct 30 19:13:58.440318 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:58.440268    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7748a000-8451-4c97-8e69-3fd63eafc781-bundle" pod "7748a000-8451-4c97-8e69-3fd63eafc781" (UID: "7748a000-8451-4c97-8e69-3fd63eafc781"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:58.445782 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:58.445750    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7748a000-8451-4c97-8e69-3fd63eafc781-kube-api-access-cvsgx" pod "7748a000-8451-4c97-8e69-3fd63eafc781" (UID: "7748a000-8451-4c97-8e69-3fd63eafc781"). InnerVolumeSpecName "kube-api-access-cvsgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:13:58.447658 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:58.447633    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7748a000-8451-4c97-8e69-3fd63eafc781-util" pod "7748a000-8451-4c97-8e69-3fd63eafc781" (UID: "7748a000-8451-4c97-8e69-3fd63eafc781"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:13:58.541527 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:58.541463    4524 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7748a000-8451-4c97-8e69-3fd63eafc781-bundle\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:58.541527 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:58.541489    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvsgx\" (UniqueName: \"kubernetes.io/projected/7748a000-8451-4c97-8e69-3fd63eafc781-kube-api-access-cvsgx\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:58.541527 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:58.541499    4524 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7748a000-8451-4c97-8e69-3fd63eafc781-util\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:13:59.281998 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:59.281963    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8"
Oct 30 19:13:59.282426 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:59.281962    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6xbzx8" event={"ID":"7748a000-8451-4c97-8e69-3fd63eafc781","Type":"ContainerDied","Data":"7dc15c4ca976c6057b5b19ffc72de7af2a53afd8d7987cd24f5019e51cf148b7"}
Oct 30 19:13:59.282426 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:13:59.282123    4524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7dc15c4ca976c6057b5b19ffc72de7af2a53afd8d7987cd24f5019e51cf148b7"
Oct 30 19:14:00.288730 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:00.288697    4524 generic.go:356] "Generic (PLEG): container finished" podID="1125a460-6a15-4179-a5fa-00aa4fc10968" containerID="8b8b02788f6fbee5195ab5dc67ffa8f0ca1dd0da1fd68dca4ec3dbc540138810" exitCode=0
Oct 30 19:14:00.289033 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:00.288765    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv" event={"ID":"1125a460-6a15-4179-a5fa-00aa4fc10968","Type":"ContainerDied","Data":"8b8b02788f6fbee5195ab5dc67ffa8f0ca1dd0da1fd68dca4ec3dbc540138810"}
Oct 30 19:14:01.295506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:01.295472    4524 generic.go:356] "Generic (PLEG): container finished" podID="1125a460-6a15-4179-a5fa-00aa4fc10968" containerID="57995c8f0c988962218265862401c05cb9a297899412124811a2b2dfb8a557ba" exitCode=0
Oct 30 19:14:01.295893 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:01.295557    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv" event={"ID":"1125a460-6a15-4179-a5fa-00aa4fc10968","Type":"ContainerDied","Data":"57995c8f0c988962218265862401c05cb9a297899412124811a2b2dfb8a557ba"}
Oct 30 19:14:02.427828 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:02.427800    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv"
Oct 30 19:14:02.469668 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:02.469630    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1125a460-6a15-4179-a5fa-00aa4fc10968-util\" (UniqueName: \"kubernetes.io/empty-dir/1125a460-6a15-4179-a5fa-00aa4fc10968-util\") pod \"1125a460-6a15-4179-a5fa-00aa4fc10968\" (UID: \"1125a460-6a15-4179-a5fa-00aa4fc10968\") "
Oct 30 19:14:02.469841 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:02.469759    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1125a460-6a15-4179-a5fa-00aa4fc10968-kube-api-access-psch8\" (UniqueName: \"kubernetes.io/projected/1125a460-6a15-4179-a5fa-00aa4fc10968-kube-api-access-psch8\") pod \"1125a460-6a15-4179-a5fa-00aa4fc10968\" (UID: \"1125a460-6a15-4179-a5fa-00aa4fc10968\") "
Oct 30 19:14:02.469841 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:02.469821    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1125a460-6a15-4179-a5fa-00aa4fc10968-bundle\" (UniqueName: \"kubernetes.io/empty-dir/1125a460-6a15-4179-a5fa-00aa4fc10968-bundle\") pod \"1125a460-6a15-4179-a5fa-00aa4fc10968\" (UID: \"1125a460-6a15-4179-a5fa-00aa4fc10968\") "
Oct 30 19:14:02.470541 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:02.470514    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1125a460-6a15-4179-a5fa-00aa4fc10968-bundle" pod "1125a460-6a15-4179-a5fa-00aa4fc10968" (UID: "1125a460-6a15-4179-a5fa-00aa4fc10968"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:02.471954 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:02.471934    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1125a460-6a15-4179-a5fa-00aa4fc10968-kube-api-access-psch8" pod "1125a460-6a15-4179-a5fa-00aa4fc10968" (UID: "1125a460-6a15-4179-a5fa-00aa4fc10968"). InnerVolumeSpecName "kube-api-access-psch8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:02.475701 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:02.475676    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1125a460-6a15-4179-a5fa-00aa4fc10968-util" pod "1125a460-6a15-4179-a5fa-00aa4fc10968" (UID: "1125a460-6a15-4179-a5fa-00aa4fc10968"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:02.570566 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:02.570472    4524 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1125a460-6a15-4179-a5fa-00aa4fc10968-util\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:02.570566 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:02.570515    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psch8\" (UniqueName: \"kubernetes.io/projected/1125a460-6a15-4179-a5fa-00aa4fc10968-kube-api-access-psch8\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:02.570566 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:02.570527    4524 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1125a460-6a15-4179-a5fa-00aa4fc10968-bundle\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:03.306470 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:03.306430    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv" event={"ID":"1125a460-6a15-4179-a5fa-00aa4fc10968","Type":"ContainerDied","Data":"e385a51f5db390569804686c2c41304d7dac8f01dd3e3f5d2982a574701f9d55"}
Oct 30 19:14:03.306470 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:03.306471    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4czmcqv"
Oct 30 19:14:03.306680 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:03.306488    4524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e385a51f5db390569804686c2c41304d7dac8f01dd3e3f5d2982a574701f9d55"
Oct 30 19:14:11.990002 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.989967    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"]
Oct 30 19:14:11.990462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990329    4524 cpu_manager.go:414] "containerMap: removing container" podUID="6340301b-511c-4c80-bae8-1bd4582b2d65" containerName="util"
Oct 30 19:14:11.990462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990342    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6340301b-511c-4c80-bae8-1bd4582b2d65" containerName="util"
Oct 30 19:14:11.990462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990354    4524 cpu_manager.go:414] "containerMap: removing container" podUID="6340301b-511c-4c80-bae8-1bd4582b2d65" containerName="pull"
Oct 30 19:14:11.990462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990359    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6340301b-511c-4c80-bae8-1bd4582b2d65" containerName="pull"
Oct 30 19:14:11.990462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990368    4524 cpu_manager.go:414] "containerMap: removing container" podUID="7748a000-8451-4c97-8e69-3fd63eafc781" containerName="pull"
Oct 30 19:14:11.990462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990373    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7748a000-8451-4c97-8e69-3fd63eafc781" containerName="pull"
Oct 30 19:14:11.990462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990391    4524 cpu_manager.go:414] "containerMap: removing container" podUID="7748a000-8451-4c97-8e69-3fd63eafc781" containerName="extract"
Oct 30 19:14:11.990462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990396    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7748a000-8451-4c97-8e69-3fd63eafc781" containerName="extract"
Oct 30 19:14:11.990462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990405    4524 cpu_manager.go:414] "containerMap: removing container" podUID="1125a460-6a15-4179-a5fa-00aa4fc10968" containerName="extract"
Oct 30 19:14:11.990462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990410    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1125a460-6a15-4179-a5fa-00aa4fc10968" containerName="extract"
Oct 30 19:14:11.990462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990420    4524 cpu_manager.go:414] "containerMap: removing container" podUID="97e76769-ac02-408e-8c04-10bb5c7ff179" containerName="util"
Oct 30 19:14:11.990462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990424    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97e76769-ac02-408e-8c04-10bb5c7ff179" containerName="util"
Oct 30 19:14:11.990462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990429    4524 cpu_manager.go:414] "containerMap: removing container" podUID="7748a000-8451-4c97-8e69-3fd63eafc781" containerName="util"
Oct 30 19:14:11.990462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990434    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7748a000-8451-4c97-8e69-3fd63eafc781" containerName="util"
Oct 30 19:14:11.990462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990439    4524 cpu_manager.go:414] "containerMap: removing container" podUID="97e76769-ac02-408e-8c04-10bb5c7ff179" containerName="pull"
Oct 30 19:14:11.990462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990443    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97e76769-ac02-408e-8c04-10bb5c7ff179" containerName="pull"
Oct 30 19:14:11.990462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990450    4524 cpu_manager.go:414] "containerMap: removing container" podUID="6340301b-511c-4c80-bae8-1bd4582b2d65" containerName="extract"
Oct 30 19:14:11.990462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990454    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6340301b-511c-4c80-bae8-1bd4582b2d65" containerName="extract"
Oct 30 19:14:11.990462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990469    4524 cpu_manager.go:414] "containerMap: removing container" podUID="1125a460-6a15-4179-a5fa-00aa4fc10968" containerName="pull"
Oct 30 19:14:11.990462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990474    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1125a460-6a15-4179-a5fa-00aa4fc10968" containerName="pull"
Oct 30 19:14:11.991211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990485    4524 cpu_manager.go:414] "containerMap: removing container" podUID="1125a460-6a15-4179-a5fa-00aa4fc10968" containerName="util"
Oct 30 19:14:11.991211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990490    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1125a460-6a15-4179-a5fa-00aa4fc10968" containerName="util"
Oct 30 19:14:11.991211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990496    4524 cpu_manager.go:414] "containerMap: removing container" podUID="97e76769-ac02-408e-8c04-10bb5c7ff179" containerName="extract"
Oct 30 19:14:11.991211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990500    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97e76769-ac02-408e-8c04-10bb5c7ff179" containerName="extract"
Oct 30 19:14:11.991211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990556    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="97e76769-ac02-408e-8c04-10bb5c7ff179" containerName="extract"
Oct 30 19:14:11.991211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990567    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="7748a000-8451-4c97-8e69-3fd63eafc781" containerName="extract"
Oct 30 19:14:11.991211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990573    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1125a460-6a15-4179-a5fa-00aa4fc10968" containerName="extract"
Oct 30 19:14:11.991211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.990581    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="6340301b-511c-4c80-bae8-1bd4582b2d65" containerName="extract"
Oct 30 19:14:11.993712 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:11.993693    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"
Oct 30 19:14:12.001638 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:12.001615    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 19:14:12.002012 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:12.001993    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 19:14:12.002210 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:12.002194    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6hscz\""
Oct 30 19:14:12.017563 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:12.017540    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"]
Oct 30 19:14:12.044344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:12.044313    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdvb5\" (UniqueName: \"kubernetes.io/projected/12ae6e8b-4288-4a87-a864-639e749fff2c-kube-api-access-gdvb5\") pod \"kuadrant-operator-controller-manager-6b6fb56846-kprgc\" (UID: \"12ae6e8b-4288-4a87-a864-639e749fff2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"
Oct 30 19:14:12.044344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:12.044343    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12ae6e8b-4288-4a87-a864-639e749fff2c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-kprgc\" (UID: \"12ae6e8b-4288-4a87-a864-639e749fff2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"
Oct 30 19:14:12.145726 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:12.145694    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdvb5\" (UniqueName: \"kubernetes.io/projected/12ae6e8b-4288-4a87-a864-639e749fff2c-kube-api-access-gdvb5\") pod \"kuadrant-operator-controller-manager-6b6fb56846-kprgc\" (UID: \"12ae6e8b-4288-4a87-a864-639e749fff2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"
Oct 30 19:14:12.145891 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:12.145733    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12ae6e8b-4288-4a87-a864-639e749fff2c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-kprgc\" (UID: \"12ae6e8b-4288-4a87-a864-639e749fff2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"
Oct 30 19:14:12.146100 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:12.146084    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12ae6e8b-4288-4a87-a864-639e749fff2c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-kprgc\" (UID: \"12ae6e8b-4288-4a87-a864-639e749fff2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"
Oct 30 19:14:12.158322 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:12.158296    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdvb5\" (UniqueName: \"kubernetes.io/projected/12ae6e8b-4288-4a87-a864-639e749fff2c-kube-api-access-gdvb5\") pod \"kuadrant-operator-controller-manager-6b6fb56846-kprgc\" (UID: \"12ae6e8b-4288-4a87-a864-639e749fff2c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"
Oct 30 19:14:12.306973 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:12.306885    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"
Oct 30 19:14:12.588680 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:12.588631    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"]
Oct 30 19:14:12.591019 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:14:12.590994    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12ae6e8b_4288_4a87_a864_639e749fff2c.slice/crio-d8e9278b234501922b452ecdd4d66f321edf36de13e19248f5ae829caad832d0 WatchSource:0}: Error finding container d8e9278b234501922b452ecdd4d66f321edf36de13e19248f5ae829caad832d0: Status 404 returned error can't find the container with id d8e9278b234501922b452ecdd4d66f321edf36de13e19248f5ae829caad832d0
Oct 30 19:14:13.353052 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:13.353015    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc" event={"ID":"12ae6e8b-4288-4a87-a864-639e749fff2c","Type":"ContainerStarted","Data":"d8e9278b234501922b452ecdd4d66f321edf36de13e19248f5ae829caad832d0"}
Oct 30 19:14:15.623540 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:14:15.623502    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:14:19.390822 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:19.390779    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-v9g7j"]
Oct 30 19:14:19.400475 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:19.400452    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-v9g7j"
Oct 30 19:14:19.405260 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:19.405232    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-nr6d4\""
Oct 30 19:14:19.413850 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:19.413826    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Oct 30 19:14:19.420512 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:19.420485    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-v9g7j"]
Oct 30 19:14:19.504147 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:19.504109    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdcvq\" (UniqueName: \"kubernetes.io/projected/201b4047-ff95-46c1-a3e2-c26c90093f6b-kube-api-access-bdcvq\") pod \"dns-operator-controller-manager-65bfd79598-v9g7j\" (UID: \"201b4047-ff95-46c1-a3e2-c26c90093f6b\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-v9g7j"
Oct 30 19:14:19.604680 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:19.604650    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdcvq\" (UniqueName: \"kubernetes.io/projected/201b4047-ff95-46c1-a3e2-c26c90093f6b-kube-api-access-bdcvq\") pod \"dns-operator-controller-manager-65bfd79598-v9g7j\" (UID: \"201b4047-ff95-46c1-a3e2-c26c90093f6b\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-v9g7j"
Oct 30 19:14:19.618909 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:19.618857    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdcvq\" (UniqueName: \"kubernetes.io/projected/201b4047-ff95-46c1-a3e2-c26c90093f6b-kube-api-access-bdcvq\") pod \"dns-operator-controller-manager-65bfd79598-v9g7j\" (UID: \"201b4047-ff95-46c1-a3e2-c26c90093f6b\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-v9g7j"
Oct 30 19:14:19.727041 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:19.726973    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-v9g7j"
Oct 30 19:14:20.017949 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:20.017801    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-v9g7j"]
Oct 30 19:14:20.020611 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:14:20.020585    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod201b4047_ff95_46c1_a3e2_c26c90093f6b.slice/crio-5f6327c84f6e76c67a304c1649c576b057570c16e7471a1018b77c8aba311fee WatchSource:0}: Error finding container 5f6327c84f6e76c67a304c1649c576b057570c16e7471a1018b77c8aba311fee: Status 404 returned error can't find the container with id 5f6327c84f6e76c67a304c1649c576b057570c16e7471a1018b77c8aba311fee
Oct 30 19:14:20.396015 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:20.395976    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-v9g7j" event={"ID":"201b4047-ff95-46c1-a3e2-c26c90093f6b","Type":"ContainerStarted","Data":"5f6327c84f6e76c67a304c1649c576b057570c16e7471a1018b77c8aba311fee"}
Oct 30 19:14:22.409605 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:22.409532    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc" event={"ID":"12ae6e8b-4288-4a87-a864-639e749fff2c","Type":"ContainerStarted","Data":"fd4582b677e5290db1fef43d5352ba078b9a2d582881784ca1ba3cee4f8b2a97"}
Oct 30 19:14:22.410077 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:22.409656    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"
Oct 30 19:14:22.431277 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:22.431231    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc" podStartSLOduration=2.470394944 podStartE2EDuration="11.43121842s" totalImagesPullingTime="8.960823476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:22.42989685 +0000 UTC m=+494.170887930" watchObservedRunningTime="2026-10-30 19:14:22.43121842 +0000 UTC m=+494.172209499"
Oct 30 19:14:23.416471 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:23.416432    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-v9g7j" event={"ID":"201b4047-ff95-46c1-a3e2-c26c90093f6b","Type":"ContainerStarted","Data":"2bff88afe67b33a95c7cf3c13d7f2fba2477f5fda9bb0e951651b8f87861f2c4"}
Oct 30 19:14:23.416885 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:23.416693    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-v9g7j"
Oct 30 19:14:23.437805 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:23.437754    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-v9g7j" podStartSLOduration=1.962312073 podStartE2EDuration="4.437738822s" totalImagesPullingTime="2.475426749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:23.435696196 +0000 UTC m=+495.176687274" watchObservedRunningTime="2026-10-30 19:14:23.437738822 +0000 UTC m=+495.178729900"
Oct 30 19:14:33.419194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:33.419135    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"
Oct 30 19:14:34.423344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:34.423314    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-v9g7j"
Oct 30 19:14:35.352047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:35.352014    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl"]
Oct 30 19:14:35.356659 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:35.356642    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl"
Oct 30 19:14:35.374129 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:35.374101    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl"]
Oct 30 19:14:35.538440 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:35.538405    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxjcn\" (UniqueName: \"kubernetes.io/projected/4ca54ae6-79e6-450e-9ecc-b68665d25ce5-kube-api-access-gxjcn\") pod \"kuadrant-operator-controller-manager-956956bc4-bt8bl\" (UID: \"4ca54ae6-79e6-450e-9ecc-b68665d25ce5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl"
Oct 30 19:14:35.538807 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:35.538530    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ca54ae6-79e6-450e-9ecc-b68665d25ce5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-956956bc4-bt8bl\" (UID: \"4ca54ae6-79e6-450e-9ecc-b68665d25ce5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl"
Oct 30 19:14:35.639227 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:35.639153    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ca54ae6-79e6-450e-9ecc-b68665d25ce5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-956956bc4-bt8bl\" (UID: \"4ca54ae6-79e6-450e-9ecc-b68665d25ce5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl"
Oct 30 19:14:35.639227 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:35.639211    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxjcn\" (UniqueName: \"kubernetes.io/projected/4ca54ae6-79e6-450e-9ecc-b68665d25ce5-kube-api-access-gxjcn\") pod \"kuadrant-operator-controller-manager-956956bc4-bt8bl\" (UID: \"4ca54ae6-79e6-450e-9ecc-b68665d25ce5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl"
Oct 30 19:14:35.639586 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:35.639568    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ca54ae6-79e6-450e-9ecc-b68665d25ce5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-956956bc4-bt8bl\" (UID: \"4ca54ae6-79e6-450e-9ecc-b68665d25ce5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl"
Oct 30 19:14:35.658735 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:35.658707    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxjcn\" (UniqueName: \"kubernetes.io/projected/4ca54ae6-79e6-450e-9ecc-b68665d25ce5-kube-api-access-gxjcn\") pod \"kuadrant-operator-controller-manager-956956bc4-bt8bl\" (UID: \"4ca54ae6-79e6-450e-9ecc-b68665d25ce5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl"
Oct 30 19:14:35.669563 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:35.669542    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl"
Oct 30 19:14:35.954505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:35.954471    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl"]
Oct 30 19:14:35.956021 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:14:35.955991    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ca54ae6_79e6_450e_9ecc_b68665d25ce5.slice/crio-2e2a35bf014ea3174a73ad1ad50e835de0c9cf9ae85df17d839201f3a24def3d WatchSource:0}: Error finding container 2e2a35bf014ea3174a73ad1ad50e835de0c9cf9ae85df17d839201f3a24def3d: Status 404 returned error can't find the container with id 2e2a35bf014ea3174a73ad1ad50e835de0c9cf9ae85df17d839201f3a24def3d
Oct 30 19:14:36.476892 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:36.476848    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl" event={"ID":"4ca54ae6-79e6-450e-9ecc-b68665d25ce5","Type":"ContainerStarted","Data":"61945ce0e6710e6b0a2f02723e527617020c52b2a7ecc0c36a54da3a85ab3a06"}
Oct 30 19:14:36.477074 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:36.476907    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl" event={"ID":"4ca54ae6-79e6-450e-9ecc-b68665d25ce5","Type":"ContainerStarted","Data":"2e2a35bf014ea3174a73ad1ad50e835de0c9cf9ae85df17d839201f3a24def3d"}
Oct 30 19:14:36.477074 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:36.476932    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl"
Oct 30 19:14:36.498554 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:36.498508    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl" podStartSLOduration=1.498495361 podStartE2EDuration="1.498495361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:36.497627788 +0000 UTC m=+508.238618882" watchObservedRunningTime="2026-10-30 19:14:36.498495361 +0000 UTC m=+508.239486439"
Oct 30 19:14:37.155276 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.155243    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"]
Oct 30 19:14:37.155741 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.155428    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc" podUID="12ae6e8b-4288-4a87-a864-639e749fff2c" containerName="manager" containerID="cri-o://fd4582b677e5290db1fef43d5352ba078b9a2d582881784ca1ba3cee4f8b2a97" gracePeriod=2
Oct 30 19:14:37.158359 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:14:37.158327    4524 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-kprgc\" is forbidden: User \"system:node:ip-10-0-137-200.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-200.us-east-2.compute.internal' and this object" podUID="12ae6e8b-4288-4a87-a864-639e749fff2c" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"
Oct 30 19:14:37.161048 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.161025    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"]
Oct 30 19:14:37.186249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.186220    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh"]
Oct 30 19:14:37.186719 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.186702    4524 cpu_manager.go:414] "containerMap: removing container" podUID="12ae6e8b-4288-4a87-a864-639e749fff2c" containerName="manager"
Oct 30 19:14:37.186762 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.186724    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12ae6e8b-4288-4a87-a864-639e749fff2c" containerName="manager"
Oct 30 19:14:37.186847 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.186836    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="12ae6e8b-4288-4a87-a864-639e749fff2c" containerName="manager"
Oct 30 19:14:37.190111 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.190089    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl"]
Oct 30 19:14:37.190259 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.190231    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh"
Oct 30 19:14:37.192654 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.192636    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl"]
Oct 30 19:14:37.219074 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.219051    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh"]
Oct 30 19:14:37.220427 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.220406    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2"]
Oct 30 19:14:37.220890 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.220873    4524 cpu_manager.go:414] "containerMap: removing container" podUID="4ca54ae6-79e6-450e-9ecc-b68665d25ce5" containerName="manager"
Oct 30 19:14:37.220928 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.220895    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ca54ae6-79e6-450e-9ecc-b68665d25ce5" containerName="manager"
Oct 30 19:14:37.220993 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.220982    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ca54ae6-79e6-450e-9ecc-b68665d25ce5" containerName="manager"
Oct 30 19:14:37.224238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.224220    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2"
Oct 30 19:14:37.242728 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:14:37.242681    4524 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-kprgc\" is forbidden: User \"system:node:ip-10-0-137-200.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-200.us-east-2.compute.internal' and this object" podUID="12ae6e8b-4288-4a87-a864-639e749fff2c" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"
Oct 30 19:14:37.245371 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.245349    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2"]
Oct 30 19:14:37.245534 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:14:37.245472    4524 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-kprgc\" is forbidden: User \"system:node:ip-10-0-137-200.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-200.us-east-2.compute.internal' and this object" podUID="12ae6e8b-4288-4a87-a864-639e749fff2c" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"
Oct 30 19:14:37.354660 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.354629    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa81aba7-e671-451f-9151-a1d4735ef812-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-956956bc4-7pwk2\" (UID: \"aa81aba7-e671-451f-9151-a1d4735ef812\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2"
Oct 30 19:14:37.354841 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.354677    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/02e7fe8d-c523-4c2f-86d6-e88e2530d20f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-wbxjh\" (UID: \"02e7fe8d-c523-4c2f-86d6-e88e2530d20f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh"
Oct 30 19:14:37.354841 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.354693    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shwv8\" (UniqueName: \"kubernetes.io/projected/aa81aba7-e671-451f-9151-a1d4735ef812-kube-api-access-shwv8\") pod \"kuadrant-operator-controller-manager-956956bc4-7pwk2\" (UID: \"aa81aba7-e671-451f-9151-a1d4735ef812\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2"
Oct 30 19:14:37.354922 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.354849    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvdk5\" (UniqueName: \"kubernetes.io/projected/02e7fe8d-c523-4c2f-86d6-e88e2530d20f-kube-api-access-mvdk5\") pod \"kuadrant-operator-controller-manager-6b6fb56846-wbxjh\" (UID: \"02e7fe8d-c523-4c2f-86d6-e88e2530d20f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh"
Oct 30 19:14:37.399150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.399127    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"
Oct 30 19:14:37.401128 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:14:37.401101    4524 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-kprgc\" is forbidden: User \"system:node:ip-10-0-137-200.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-200.us-east-2.compute.internal' and this object" podUID="12ae6e8b-4288-4a87-a864-639e749fff2c" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"
Oct 30 19:14:37.456214 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.456138    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvdk5\" (UniqueName: \"kubernetes.io/projected/02e7fe8d-c523-4c2f-86d6-e88e2530d20f-kube-api-access-mvdk5\") pod \"kuadrant-operator-controller-manager-6b6fb56846-wbxjh\" (UID: \"02e7fe8d-c523-4c2f-86d6-e88e2530d20f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh"
Oct 30 19:14:37.456296 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.456219    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa81aba7-e671-451f-9151-a1d4735ef812-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-956956bc4-7pwk2\" (UID: \"aa81aba7-e671-451f-9151-a1d4735ef812\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2"
Oct 30 19:14:37.456296 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.456254    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/02e7fe8d-c523-4c2f-86d6-e88e2530d20f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-wbxjh\" (UID: \"02e7fe8d-c523-4c2f-86d6-e88e2530d20f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh"
Oct 30 19:14:37.456296 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.456275    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shwv8\" (UniqueName: \"kubernetes.io/projected/aa81aba7-e671-451f-9151-a1d4735ef812-kube-api-access-shwv8\") pod \"kuadrant-operator-controller-manager-956956bc4-7pwk2\" (UID: \"aa81aba7-e671-451f-9151-a1d4735ef812\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2"
Oct 30 19:14:37.456640 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.456622    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa81aba7-e671-451f-9151-a1d4735ef812-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-956956bc4-7pwk2\" (UID: \"aa81aba7-e671-451f-9151-a1d4735ef812\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2"
Oct 30 19:14:37.456676 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.456644    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/02e7fe8d-c523-4c2f-86d6-e88e2530d20f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-wbxjh\" (UID: \"02e7fe8d-c523-4c2f-86d6-e88e2530d20f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh"
Oct 30 19:14:37.468019 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.467997    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvdk5\" (UniqueName: \"kubernetes.io/projected/02e7fe8d-c523-4c2f-86d6-e88e2530d20f-kube-api-access-mvdk5\") pod \"kuadrant-operator-controller-manager-6b6fb56846-wbxjh\" (UID: \"02e7fe8d-c523-4c2f-86d6-e88e2530d20f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh"
Oct 30 19:14:37.468263 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.468245    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shwv8\" (UniqueName: \"kubernetes.io/projected/aa81aba7-e671-451f-9151-a1d4735ef812-kube-api-access-shwv8\") pod \"kuadrant-operator-controller-manager-956956bc4-7pwk2\" (UID: \"aa81aba7-e671-451f-9151-a1d4735ef812\") " pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2"
Oct 30 19:14:37.482287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.482262    4524 generic.go:356] "Generic (PLEG): container finished" podID="12ae6e8b-4288-4a87-a864-639e749fff2c" containerID="fd4582b677e5290db1fef43d5352ba078b9a2d582881784ca1ba3cee4f8b2a97" exitCode=0
Oct 30 19:14:37.482389 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.482308    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"
Oct 30 19:14:37.482389 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.482351    4524 scope.go:122] "RemoveContainer" containerID="fd4582b677e5290db1fef43d5352ba078b9a2d582881784ca1ba3cee4f8b2a97"
Oct 30 19:14:37.484343 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:14:37.484317    4524 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-kprgc\" is forbidden: User \"system:node:ip-10-0-137-200.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-200.us-east-2.compute.internal' and this object" podUID="12ae6e8b-4288-4a87-a864-639e749fff2c" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"
Oct 30 19:14:37.490269 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.490235    4524 scope.go:122] "RemoveContainer" containerID="fd4582b677e5290db1fef43d5352ba078b9a2d582881784ca1ba3cee4f8b2a97"
Oct 30 19:14:37.490515 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:14:37.490481    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd4582b677e5290db1fef43d5352ba078b9a2d582881784ca1ba3cee4f8b2a97\": container with ID starting with fd4582b677e5290db1fef43d5352ba078b9a2d582881784ca1ba3cee4f8b2a97 not found: ID does not exist" containerID="fd4582b677e5290db1fef43d5352ba078b9a2d582881784ca1ba3cee4f8b2a97"
Oct 30 19:14:37.490574 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.490521    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd4582b677e5290db1fef43d5352ba078b9a2d582881784ca1ba3cee4f8b2a97"} err="failed to get container status \"fd4582b677e5290db1fef43d5352ba078b9a2d582881784ca1ba3cee4f8b2a97\": rpc error: code = NotFound desc = could not find container \"fd4582b677e5290db1fef43d5352ba078b9a2d582881784ca1ba3cee4f8b2a97\": container with ID starting with fd4582b677e5290db1fef43d5352ba078b9a2d582881784ca1ba3cee4f8b2a97 not found: ID does not exist"
Oct 30 19:14:37.557563 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.557543    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12ae6e8b-4288-4a87-a864-639e749fff2c-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12ae6e8b-4288-4a87-a864-639e749fff2c-extensions-socket-volume\") pod \"12ae6e8b-4288-4a87-a864-639e749fff2c\" (UID: \"12ae6e8b-4288-4a87-a864-639e749fff2c\") "
Oct 30 19:14:37.557642 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.557573    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12ae6e8b-4288-4a87-a864-639e749fff2c-kube-api-access-gdvb5\" (UniqueName: \"kubernetes.io/projected/12ae6e8b-4288-4a87-a864-639e749fff2c-kube-api-access-gdvb5\") pod \"12ae6e8b-4288-4a87-a864-639e749fff2c\" (UID: \"12ae6e8b-4288-4a87-a864-639e749fff2c\") "
Oct 30 19:14:37.558069 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.558043    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12ae6e8b-4288-4a87-a864-639e749fff2c-extensions-socket-volume" pod "12ae6e8b-4288-4a87-a864-639e749fff2c" (UID: "12ae6e8b-4288-4a87-a864-639e749fff2c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:37.559515 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.559498    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12ae6e8b-4288-4a87-a864-639e749fff2c-kube-api-access-gdvb5" pod "12ae6e8b-4288-4a87-a864-639e749fff2c" (UID: "12ae6e8b-4288-4a87-a864-639e749fff2c"). InnerVolumeSpecName "kube-api-access-gdvb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:37.562727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.562710    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh"
Oct 30 19:14:37.564846 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.564825    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2"
Oct 30 19:14:37.658753 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.658727    4524 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12ae6e8b-4288-4a87-a864-639e749fff2c-extensions-socket-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:37.658854 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.658754    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdvb5\" (UniqueName: \"kubernetes.io/projected/12ae6e8b-4288-4a87-a864-639e749fff2c-kube-api-access-gdvb5\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:37.793575 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:14:37.793539    4524 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-kprgc\" is forbidden: User \"system:node:ip-10-0-137-200.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-200.us-east-2.compute.internal' and this object" podUID="12ae6e8b-4288-4a87-a864-639e749fff2c" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"
Oct 30 19:14:37.848223 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.848200    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh"]
Oct 30 19:14:37.853555 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:14:37.853530    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02e7fe8d_c523_4c2f_86d6_e88e2530d20f.slice/crio-de6ab0fc7ee504ac8fd85c2277943afcd605f216e3cf721ca660762a9a60cc51 WatchSource:0}: Error finding container de6ab0fc7ee504ac8fd85c2277943afcd605f216e3cf721ca660762a9a60cc51: Status 404 returned error can't find the container with id de6ab0fc7ee504ac8fd85c2277943afcd605f216e3cf721ca660762a9a60cc51
Oct 30 19:14:37.854363 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:14:37.854347    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa81aba7_e671_451f_9151_a1d4735ef812.slice/crio-b4423d451eaf757c2badd947fc62493649a08fc6b14af90b3307d2896ec2523e WatchSource:0}: Error finding container b4423d451eaf757c2badd947fc62493649a08fc6b14af90b3307d2896ec2523e: Status 404 returned error can't find the container with id b4423d451eaf757c2badd947fc62493649a08fc6b14af90b3307d2896ec2523e
Oct 30 19:14:37.854757 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:37.854737    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2"]
Oct 30 19:14:38.386812 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:14:38.386776    4524 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-kprgc\" is forbidden: User \"system:node:ip-10-0-137-200.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-200.us-east-2.compute.internal' and this object" podUID="12ae6e8b-4288-4a87-a864-639e749fff2c" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-kprgc"
Oct 30 19:14:38.388834 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:38.388809    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="12ae6e8b-4288-4a87-a864-639e749fff2c" path="/var/lib/kubelet/pods/12ae6e8b-4288-4a87-a864-639e749fff2c/volumes"
Oct 30 19:14:38.488592 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:38.488509    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2" event={"ID":"aa81aba7-e671-451f-9151-a1d4735ef812","Type":"ContainerStarted","Data":"6f08c41afc0135cad3c9f59bb12ddb717610c20ef047dc4ec58fa0472bd5a5da"}
Oct 30 19:14:38.488592 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:38.488567    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2" event={"ID":"aa81aba7-e671-451f-9151-a1d4735ef812","Type":"ContainerStarted","Data":"b4423d451eaf757c2badd947fc62493649a08fc6b14af90b3307d2896ec2523e"}
Oct 30 19:14:38.488780 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:38.488596    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2"
Oct 30 19:14:38.491224 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:38.491199    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh" event={"ID":"02e7fe8d-c523-4c2f-86d6-e88e2530d20f","Type":"ContainerStarted","Data":"1ed76305b4fa5c0fae4b36fd8af6b0dfd919a46b7757506968209b8c5a0f2c2c"}
Oct 30 19:14:38.491328 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:38.491241    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh" event={"ID":"02e7fe8d-c523-4c2f-86d6-e88e2530d20f","Type":"ContainerStarted","Data":"de6ab0fc7ee504ac8fd85c2277943afcd605f216e3cf721ca660762a9a60cc51"}
Oct 30 19:14:38.491328 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:38.491283    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl" podUID="4ca54ae6-79e6-450e-9ecc-b68665d25ce5" containerName="manager" containerID="cri-o://61945ce0e6710e6b0a2f02723e527617020c52b2a7ecc0c36a54da3a85ab3a06" gracePeriod=2
Oct 30 19:14:38.513359 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:38.512882    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2" podStartSLOduration=1.5128673350000001 podStartE2EDuration="1.512867335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:38.512371036 +0000 UTC m=+510.253362105" watchObservedRunningTime="2026-10-30 19:14:38.512867335 +0000 UTC m=+510.253858413"
Oct 30 19:14:38.536304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:38.536259    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh" podStartSLOduration=1.536243155 podStartE2EDuration="1.536243155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:38.534286647 +0000 UTC m=+510.275277725" watchObservedRunningTime="2026-10-30 19:14:38.536243155 +0000 UTC m=+510.277234234"
Oct 30 19:14:38.730937 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:38.730912    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl"
Oct 30 19:14:38.869222 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:38.869190    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ca54ae6-79e6-450e-9ecc-b68665d25ce5-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ca54ae6-79e6-450e-9ecc-b68665d25ce5-extensions-socket-volume\") pod \"4ca54ae6-79e6-450e-9ecc-b68665d25ce5\" (UID: \"4ca54ae6-79e6-450e-9ecc-b68665d25ce5\") "
Oct 30 19:14:38.869383 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:38.869340    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4ca54ae6-79e6-450e-9ecc-b68665d25ce5-kube-api-access-gxjcn\" (UniqueName: \"kubernetes.io/projected/4ca54ae6-79e6-450e-9ecc-b68665d25ce5-kube-api-access-gxjcn\") pod \"4ca54ae6-79e6-450e-9ecc-b68665d25ce5\" (UID: \"4ca54ae6-79e6-450e-9ecc-b68665d25ce5\") "
Oct 30 19:14:38.869657 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:38.869629    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca54ae6-79e6-450e-9ecc-b68665d25ce5-extensions-socket-volume" pod "4ca54ae6-79e6-450e-9ecc-b68665d25ce5" (UID: "4ca54ae6-79e6-450e-9ecc-b68665d25ce5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:38.874297 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:38.874269    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ca54ae6-79e6-450e-9ecc-b68665d25ce5-kube-api-access-gxjcn" pod "4ca54ae6-79e6-450e-9ecc-b68665d25ce5" (UID: "4ca54ae6-79e6-450e-9ecc-b68665d25ce5"). InnerVolumeSpecName "kube-api-access-gxjcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:38.970298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:38.970267    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxjcn\" (UniqueName: \"kubernetes.io/projected/4ca54ae6-79e6-450e-9ecc-b68665d25ce5-kube-api-access-gxjcn\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:38.970298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:38.970299    4524 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ca54ae6-79e6-450e-9ecc-b68665d25ce5-extensions-socket-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:39.497876 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:39.497844    4524 generic.go:356] "Generic (PLEG): container finished" podID="4ca54ae6-79e6-450e-9ecc-b68665d25ce5" containerID="61945ce0e6710e6b0a2f02723e527617020c52b2a7ecc0c36a54da3a85ab3a06" exitCode=0
Oct 30 19:14:39.498321 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:39.497884    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl"
Oct 30 19:14:39.498321 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:39.497939    4524 scope.go:122] "RemoveContainer" containerID="61945ce0e6710e6b0a2f02723e527617020c52b2a7ecc0c36a54da3a85ab3a06"
Oct 30 19:14:39.498321 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:39.498116    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh"
Oct 30 19:14:39.501802 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:14:39.501776    4524 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-956956bc4-bt8bl\" is forbidden: User \"system:node:ip-10-0-137-200.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-200.us-east-2.compute.internal' and this object" podUID="4ca54ae6-79e6-450e-9ecc-b68665d25ce5" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl"
Oct 30 19:14:39.509327 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:39.509313    4524 scope.go:122] "RemoveContainer" containerID="61945ce0e6710e6b0a2f02723e527617020c52b2a7ecc0c36a54da3a85ab3a06"
Oct 30 19:14:39.509765 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:14:39.509742    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61945ce0e6710e6b0a2f02723e527617020c52b2a7ecc0c36a54da3a85ab3a06\": container with ID starting with 61945ce0e6710e6b0a2f02723e527617020c52b2a7ecc0c36a54da3a85ab3a06 not found: ID does not exist" containerID="61945ce0e6710e6b0a2f02723e527617020c52b2a7ecc0c36a54da3a85ab3a06"
Oct 30 19:14:39.509818 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:39.509776    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61945ce0e6710e6b0a2f02723e527617020c52b2a7ecc0c36a54da3a85ab3a06"} err="failed to get container status \"61945ce0e6710e6b0a2f02723e527617020c52b2a7ecc0c36a54da3a85ab3a06\": rpc error: code = NotFound desc = could not find container \"61945ce0e6710e6b0a2f02723e527617020c52b2a7ecc0c36a54da3a85ab3a06\": container with ID starting with 61945ce0e6710e6b0a2f02723e527617020c52b2a7ecc0c36a54da3a85ab3a06 not found: ID does not exist"
Oct 30 19:14:39.510813 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:14:39.510784    4524 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-956956bc4-bt8bl\" is forbidden: User \"system:node:ip-10-0-137-200.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-200.us-east-2.compute.internal' and this object" podUID="4ca54ae6-79e6-450e-9ecc-b68665d25ce5" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-bt8bl"
Oct 30 19:14:40.388037 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:40.388003    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4ca54ae6-79e6-450e-9ecc-b68665d25ce5" path="/var/lib/kubelet/pods/4ca54ae6-79e6-450e-9ecc-b68665d25ce5/volumes"
Oct 30 19:14:45.623669 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:14:45.623629    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:14:49.500532 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:49.500505    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2"
Oct 30 19:14:49.565040 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:49.565010    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh"]
Oct 30 19:14:49.565301 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:49.565275    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh" podUID="02e7fe8d-c523-4c2f-86d6-e88e2530d20f" containerName="manager" containerID="cri-o://1ed76305b4fa5c0fae4b36fd8af6b0dfd919a46b7757506968209b8c5a0f2c2c" gracePeriod=10
Oct 30 19:14:49.567232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:49.567208    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh"
Oct 30 19:14:49.843902 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:49.843876    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh"
Oct 30 19:14:49.877818 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:49.877789    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q"]
Oct 30 19:14:49.878205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:49.878186    4524 cpu_manager.go:414] "containerMap: removing container" podUID="02e7fe8d-c523-4c2f-86d6-e88e2530d20f" containerName="manager"
Oct 30 19:14:49.878205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:49.878204    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="02e7fe8d-c523-4c2f-86d6-e88e2530d20f" containerName="manager"
Oct 30 19:14:49.878342 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:49.878266    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="02e7fe8d-c523-4c2f-86d6-e88e2530d20f" containerName="manager"
Oct 30 19:14:49.897866 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:49.897846    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q"]
Oct 30 19:14:49.897987 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:49.897926    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q"
Oct 30 19:14:49.940720 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:49.940696    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02e7fe8d-c523-4c2f-86d6-e88e2530d20f-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/02e7fe8d-c523-4c2f-86d6-e88e2530d20f-extensions-socket-volume\") pod \"02e7fe8d-c523-4c2f-86d6-e88e2530d20f\" (UID: \"02e7fe8d-c523-4c2f-86d6-e88e2530d20f\") "
Oct 30 19:14:49.940823 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:49.940799    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdm66\" (UniqueName: \"kubernetes.io/projected/2c4e2402-4877-4386-a104-58050b5c0d09-kube-api-access-mdm66\") pod \"kuadrant-operator-controller-manager-57bb9bfff-vxz4q\" (UID: \"2c4e2402-4877-4386-a104-58050b5c0d09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q"
Oct 30 19:14:49.940881 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:49.940824    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c4e2402-4877-4386-a104-58050b5c0d09-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-vxz4q\" (UID: \"2c4e2402-4877-4386-a104-58050b5c0d09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q"
Oct 30 19:14:49.941060 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:49.941038    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02e7fe8d-c523-4c2f-86d6-e88e2530d20f-extensions-socket-volume" pod "02e7fe8d-c523-4c2f-86d6-e88e2530d20f" (UID: "02e7fe8d-c523-4c2f-86d6-e88e2530d20f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:14:50.041363 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:50.041338    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/02e7fe8d-c523-4c2f-86d6-e88e2530d20f-kube-api-access-mvdk5\" (UniqueName: \"kubernetes.io/projected/02e7fe8d-c523-4c2f-86d6-e88e2530d20f-kube-api-access-mvdk5\") pod \"02e7fe8d-c523-4c2f-86d6-e88e2530d20f\" (UID: \"02e7fe8d-c523-4c2f-86d6-e88e2530d20f\") "
Oct 30 19:14:50.041488 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:50.041458    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdm66\" (UniqueName: \"kubernetes.io/projected/2c4e2402-4877-4386-a104-58050b5c0d09-kube-api-access-mdm66\") pod \"kuadrant-operator-controller-manager-57bb9bfff-vxz4q\" (UID: \"2c4e2402-4877-4386-a104-58050b5c0d09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q"
Oct 30 19:14:50.041488 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:50.041482    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c4e2402-4877-4386-a104-58050b5c0d09-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-vxz4q\" (UID: \"2c4e2402-4877-4386-a104-58050b5c0d09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q"
Oct 30 19:14:50.041576 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:50.041544    4524 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/02e7fe8d-c523-4c2f-86d6-e88e2530d20f-extensions-socket-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:50.041806 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:50.041790    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c4e2402-4877-4386-a104-58050b5c0d09-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-vxz4q\" (UID: \"2c4e2402-4877-4386-a104-58050b5c0d09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q"
Oct 30 19:14:50.043211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:50.043189    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02e7fe8d-c523-4c2f-86d6-e88e2530d20f-kube-api-access-mvdk5" pod "02e7fe8d-c523-4c2f-86d6-e88e2530d20f" (UID: "02e7fe8d-c523-4c2f-86d6-e88e2530d20f"). InnerVolumeSpecName "kube-api-access-mvdk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:14:50.060837 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:50.060785    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdm66\" (UniqueName: \"kubernetes.io/projected/2c4e2402-4877-4386-a104-58050b5c0d09-kube-api-access-mdm66\") pod \"kuadrant-operator-controller-manager-57bb9bfff-vxz4q\" (UID: \"2c4e2402-4877-4386-a104-58050b5c0d09\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q"
Oct 30 19:14:50.141893 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:50.141871    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvdk5\" (UniqueName: \"kubernetes.io/projected/02e7fe8d-c523-4c2f-86d6-e88e2530d20f-kube-api-access-mvdk5\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:14:50.220852 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:50.220833    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q"
Oct 30 19:14:50.529142 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:50.529026    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q"]
Oct 30 19:14:50.532000 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:14:50.531971    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c4e2402_4877_4386_a104_58050b5c0d09.slice/crio-9bca1dc2cba469683d1585ed469c201c603e7d80af65777b038da17696be8d0d WatchSource:0}: Error finding container 9bca1dc2cba469683d1585ed469c201c603e7d80af65777b038da17696be8d0d: Status 404 returned error can't find the container with id 9bca1dc2cba469683d1585ed469c201c603e7d80af65777b038da17696be8d0d
Oct 30 19:14:50.553738 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:50.553714    4524 generic.go:356] "Generic (PLEG): container finished" podID="02e7fe8d-c523-4c2f-86d6-e88e2530d20f" containerID="1ed76305b4fa5c0fae4b36fd8af6b0dfd919a46b7757506968209b8c5a0f2c2c" exitCode=0
Oct 30 19:14:50.553830 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:50.553753    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh" event={"ID":"02e7fe8d-c523-4c2f-86d6-e88e2530d20f","Type":"ContainerDied","Data":"1ed76305b4fa5c0fae4b36fd8af6b0dfd919a46b7757506968209b8c5a0f2c2c"}
Oct 30 19:14:50.553830 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:50.553803    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh" event={"ID":"02e7fe8d-c523-4c2f-86d6-e88e2530d20f","Type":"ContainerDied","Data":"de6ab0fc7ee504ac8fd85c2277943afcd605f216e3cf721ca660762a9a60cc51"}
Oct 30 19:14:50.553830 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:50.553824    4524 scope.go:122] "RemoveContainer" containerID="1ed76305b4fa5c0fae4b36fd8af6b0dfd919a46b7757506968209b8c5a0f2c2c"
Oct 30 19:14:50.553957 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:50.553761    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh"
Oct 30 19:14:50.555346 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:50.555324    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q" event={"ID":"2c4e2402-4877-4386-a104-58050b5c0d09","Type":"ContainerStarted","Data":"9bca1dc2cba469683d1585ed469c201c603e7d80af65777b038da17696be8d0d"}
Oct 30 19:14:50.567068 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:50.566902    4524 scope.go:122] "RemoveContainer" containerID="1ed76305b4fa5c0fae4b36fd8af6b0dfd919a46b7757506968209b8c5a0f2c2c"
Oct 30 19:14:50.567231 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:14:50.567212    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ed76305b4fa5c0fae4b36fd8af6b0dfd919a46b7757506968209b8c5a0f2c2c\": container with ID starting with 1ed76305b4fa5c0fae4b36fd8af6b0dfd919a46b7757506968209b8c5a0f2c2c not found: ID does not exist" containerID="1ed76305b4fa5c0fae4b36fd8af6b0dfd919a46b7757506968209b8c5a0f2c2c"
Oct 30 19:14:50.567314 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:50.567238    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ed76305b4fa5c0fae4b36fd8af6b0dfd919a46b7757506968209b8c5a0f2c2c"} err="failed to get container status \"1ed76305b4fa5c0fae4b36fd8af6b0dfd919a46b7757506968209b8c5a0f2c2c\": rpc error: code = NotFound desc = could not find container \"1ed76305b4fa5c0fae4b36fd8af6b0dfd919a46b7757506968209b8c5a0f2c2c\": container with ID starting with 1ed76305b4fa5c0fae4b36fd8af6b0dfd919a46b7757506968209b8c5a0f2c2c not found: ID does not exist"
Oct 30 19:14:50.577429 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:50.577408    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh"]
Oct 30 19:14:50.581822 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:50.581802    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-wbxjh"]
Oct 30 19:14:51.562911 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:51.562879    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q" event={"ID":"2c4e2402-4877-4386-a104-58050b5c0d09","Type":"ContainerStarted","Data":"8dedec230f9544e34cb6d58ff17a7c754cca4413570e78dcf77d38f020c5bceb"}
Oct 30 19:14:51.563330 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:51.563028    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q"
Oct 30 19:14:51.590704 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:51.590660    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q" podStartSLOduration=2.590649179 podStartE2EDuration="2.590649179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:51.588782933 +0000 UTC m=+523.329774012" watchObservedRunningTime="2026-10-30 19:14:51.590649179 +0000 UTC m=+523.331640258"
Oct 30 19:14:52.387311 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:14:52.387281    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="02e7fe8d-c523-4c2f-86d6-e88e2530d20f" path="/var/lib/kubelet/pods/02e7fe8d-c523-4c2f-86d6-e88e2530d20f/volumes"
Oct 30 19:15:02.569807 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:02.569775    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q"
Oct 30 19:15:02.630875 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:02.630838    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2"]
Oct 30 19:15:02.631095 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:02.631056    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2" podUID="aa81aba7-e671-451f-9151-a1d4735ef812" containerName="manager" containerID="cri-o://6f08c41afc0135cad3c9f59bb12ddb717610c20ef047dc4ec58fa0472bd5a5da" gracePeriod=10
Oct 30 19:15:02.876607 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:02.876584    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2"
Oct 30 19:15:02.926278 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:02.926243    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa81aba7-e671-451f-9151-a1d4735ef812-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa81aba7-e671-451f-9151-a1d4735ef812-extensions-socket-volume\") pod \"aa81aba7-e671-451f-9151-a1d4735ef812\" (UID: \"aa81aba7-e671-451f-9151-a1d4735ef812\") "
Oct 30 19:15:02.926439 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:02.926292    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa81aba7-e671-451f-9151-a1d4735ef812-kube-api-access-shwv8\" (UniqueName: \"kubernetes.io/projected/aa81aba7-e671-451f-9151-a1d4735ef812-kube-api-access-shwv8\") pod \"aa81aba7-e671-451f-9151-a1d4735ef812\" (UID: \"aa81aba7-e671-451f-9151-a1d4735ef812\") "
Oct 30 19:15:02.926631 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:02.926609    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa81aba7-e671-451f-9151-a1d4735ef812-extensions-socket-volume" pod "aa81aba7-e671-451f-9151-a1d4735ef812" (UID: "aa81aba7-e671-451f-9151-a1d4735ef812"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:15:02.928412 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:02.928389    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa81aba7-e671-451f-9151-a1d4735ef812-kube-api-access-shwv8" pod "aa81aba7-e671-451f-9151-a1d4735ef812" (UID: "aa81aba7-e671-451f-9151-a1d4735ef812"). InnerVolumeSpecName "kube-api-access-shwv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:03.026854 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:03.026817    4524 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa81aba7-e671-451f-9151-a1d4735ef812-extensions-socket-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:03.026854 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:03.026861    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shwv8\" (UniqueName: \"kubernetes.io/projected/aa81aba7-e671-451f-9151-a1d4735ef812-kube-api-access-shwv8\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:03.619256 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:03.619221    4524 generic.go:356] "Generic (PLEG): container finished" podID="aa81aba7-e671-451f-9151-a1d4735ef812" containerID="6f08c41afc0135cad3c9f59bb12ddb717610c20ef047dc4ec58fa0472bd5a5da" exitCode=0
Oct 30 19:15:03.619703 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:03.619263    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2" event={"ID":"aa81aba7-e671-451f-9151-a1d4735ef812","Type":"ContainerDied","Data":"6f08c41afc0135cad3c9f59bb12ddb717610c20ef047dc4ec58fa0472bd5a5da"}
Oct 30 19:15:03.619703 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:03.619276    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2"
Oct 30 19:15:03.619703 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:03.619306    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2" event={"ID":"aa81aba7-e671-451f-9151-a1d4735ef812","Type":"ContainerDied","Data":"b4423d451eaf757c2badd947fc62493649a08fc6b14af90b3307d2896ec2523e"}
Oct 30 19:15:03.619703 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:03.619329    4524 scope.go:122] "RemoveContainer" containerID="6f08c41afc0135cad3c9f59bb12ddb717610c20ef047dc4ec58fa0472bd5a5da"
Oct 30 19:15:03.628435 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:03.628416    4524 scope.go:122] "RemoveContainer" containerID="6f08c41afc0135cad3c9f59bb12ddb717610c20ef047dc4ec58fa0472bd5a5da"
Oct 30 19:15:03.628697 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:15:03.628676    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f08c41afc0135cad3c9f59bb12ddb717610c20ef047dc4ec58fa0472bd5a5da\": container with ID starting with 6f08c41afc0135cad3c9f59bb12ddb717610c20ef047dc4ec58fa0472bd5a5da not found: ID does not exist" containerID="6f08c41afc0135cad3c9f59bb12ddb717610c20ef047dc4ec58fa0472bd5a5da"
Oct 30 19:15:03.628742 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:03.628706    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f08c41afc0135cad3c9f59bb12ddb717610c20ef047dc4ec58fa0472bd5a5da"} err="failed to get container status \"6f08c41afc0135cad3c9f59bb12ddb717610c20ef047dc4ec58fa0472bd5a5da\": rpc error: code = NotFound desc = could not find container \"6f08c41afc0135cad3c9f59bb12ddb717610c20ef047dc4ec58fa0472bd5a5da\": container with ID starting with 6f08c41afc0135cad3c9f59bb12ddb717610c20ef047dc4ec58fa0472bd5a5da not found: ID does not exist"
Oct 30 19:15:03.640762 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:03.640716    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2"]
Oct 30 19:15:03.647706 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:03.647682    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-956956bc4-7pwk2"]
Oct 30 19:15:04.388384 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:04.388348    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa81aba7-e671-451f-9151-a1d4735ef812" path="/var/lib/kubelet/pods/aa81aba7-e671-451f-9151-a1d4735ef812/volumes"
Oct 30 19:15:15.623578 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:15:15.623538    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:15:18.625198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:18.625155    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-g74sq"]
Oct 30 19:15:18.625560 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:18.625496    4524 cpu_manager.go:414] "containerMap: removing container" podUID="aa81aba7-e671-451f-9151-a1d4735ef812" containerName="manager"
Oct 30 19:15:18.625560 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:18.625506    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa81aba7-e671-451f-9151-a1d4735ef812" containerName="manager"
Oct 30 19:15:18.625631 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:18.625587    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa81aba7-e671-451f-9151-a1d4735ef812" containerName="manager"
Oct 30 19:15:18.630247 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:18.630226    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-g74sq"
Oct 30 19:15:18.636210 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:18.636190    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 19:15:18.636871 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:18.636250    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zs2nd\""
Oct 30 19:15:18.638557 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:18.638538    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-g74sq"]
Oct 30 19:15:18.724504 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:18.724468    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-g74sq"]
Oct 30 19:15:18.751494 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:18.751464    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8521fe76-b5a8-4009-b878-1ed15193d2e4-config-file\") pod \"limitador-limitador-74dfcfb894-g74sq\" (UID: \"8521fe76-b5a8-4009-b878-1ed15193d2e4\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-g74sq"
Oct 30 19:15:18.751651 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:18.751517    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgwwc\" (UniqueName: \"kubernetes.io/projected/8521fe76-b5a8-4009-b878-1ed15193d2e4-kube-api-access-dgwwc\") pod \"limitador-limitador-74dfcfb894-g74sq\" (UID: \"8521fe76-b5a8-4009-b878-1ed15193d2e4\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-g74sq"
Oct 30 19:15:18.852286 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:18.852257    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8521fe76-b5a8-4009-b878-1ed15193d2e4-config-file\") pod \"limitador-limitador-74dfcfb894-g74sq\" (UID: \"8521fe76-b5a8-4009-b878-1ed15193d2e4\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-g74sq"
Oct 30 19:15:18.852426 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:18.852318    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgwwc\" (UniqueName: \"kubernetes.io/projected/8521fe76-b5a8-4009-b878-1ed15193d2e4-kube-api-access-dgwwc\") pod \"limitador-limitador-74dfcfb894-g74sq\" (UID: \"8521fe76-b5a8-4009-b878-1ed15193d2e4\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-g74sq"
Oct 30 19:15:18.852845 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:18.852826    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8521fe76-b5a8-4009-b878-1ed15193d2e4-config-file\") pod \"limitador-limitador-74dfcfb894-g74sq\" (UID: \"8521fe76-b5a8-4009-b878-1ed15193d2e4\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-g74sq"
Oct 30 19:15:18.863597 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:18.863573    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgwwc\" (UniqueName: \"kubernetes.io/projected/8521fe76-b5a8-4009-b878-1ed15193d2e4-kube-api-access-dgwwc\") pod \"limitador-limitador-74dfcfb894-g74sq\" (UID: \"8521fe76-b5a8-4009-b878-1ed15193d2e4\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-g74sq"
Oct 30 19:15:18.943531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:18.943481    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-g74sq"
Oct 30 19:15:19.215196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:19.215153    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-g74sq"]
Oct 30 19:15:19.217867 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:15:19.217841    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8521fe76_b5a8_4009_b878_1ed15193d2e4.slice/crio-23bd61269b66e5c10b456fcb3ae97d2538f2c03c7562bc419485b31ea884b3e0 WatchSource:0}: Error finding container 23bd61269b66e5c10b456fcb3ae97d2538f2c03c7562bc419485b31ea884b3e0: Status 404 returned error can't find the container with id 23bd61269b66e5c10b456fcb3ae97d2538f2c03c7562bc419485b31ea884b3e0
Oct 30 19:15:19.702511 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:19.702477    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-g74sq" event={"ID":"8521fe76-b5a8-4009-b878-1ed15193d2e4","Type":"ContainerStarted","Data":"23bd61269b66e5c10b456fcb3ae97d2538f2c03c7562bc419485b31ea884b3e0"}
Oct 30 19:15:21.712299 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:21.712263    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-g74sq" event={"ID":"8521fe76-b5a8-4009-b878-1ed15193d2e4","Type":"ContainerStarted","Data":"5816ddc8eefbddc8c6c23d54b6ecbdbded4aca4de3f42d908f9ee3f12e7b23c8"}
Oct 30 19:15:21.712662 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:21.712326    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-g74sq"
Oct 30 19:15:21.731087 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:21.731030    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-g74sq" podStartSLOduration=2.222381282 podStartE2EDuration="3.731018593s" totalImagesPullingTime="1.508637311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:21.729157553 +0000 UTC m=+553.470148631" watchObservedRunningTime="2026-10-30 19:15:21.731018593 +0000 UTC m=+553.472009672"
Oct 30 19:15:32.717634 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:32.717599    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-g74sq"
Oct 30 19:15:33.324843 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:33.324808    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-g74sq"]
Oct 30 19:15:33.325024 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:33.324988    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-g74sq" podUID="8521fe76-b5a8-4009-b878-1ed15193d2e4" containerName="limitador" containerID="cri-o://5816ddc8eefbddc8c6c23d54b6ecbdbded4aca4de3f42d908f9ee3f12e7b23c8" gracePeriod=30
Oct 30 19:15:33.768191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:33.768137    4524 generic.go:356] "Generic (PLEG): container finished" podID="8521fe76-b5a8-4009-b878-1ed15193d2e4" containerID="5816ddc8eefbddc8c6c23d54b6ecbdbded4aca4de3f42d908f9ee3f12e7b23c8" exitCode=0
Oct 30 19:15:33.768641 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:33.768213    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-g74sq" event={"ID":"8521fe76-b5a8-4009-b878-1ed15193d2e4","Type":"ContainerDied","Data":"5816ddc8eefbddc8c6c23d54b6ecbdbded4aca4de3f42d908f9ee3f12e7b23c8"}
Oct 30 19:15:33.868472 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:33.868447    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-g74sq"
Oct 30 19:15:33.962892 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:33.962820    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8521fe76-b5a8-4009-b878-1ed15193d2e4-kube-api-access-dgwwc\" (UniqueName: \"kubernetes.io/projected/8521fe76-b5a8-4009-b878-1ed15193d2e4-kube-api-access-dgwwc\") pod \"8521fe76-b5a8-4009-b878-1ed15193d2e4\" (UID: \"8521fe76-b5a8-4009-b878-1ed15193d2e4\") "
Oct 30 19:15:33.963024 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:33.962905    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8521fe76-b5a8-4009-b878-1ed15193d2e4-config-file\" (UniqueName: \"kubernetes.io/configmap/8521fe76-b5a8-4009-b878-1ed15193d2e4-config-file\") pod \"8521fe76-b5a8-4009-b878-1ed15193d2e4\" (UID: \"8521fe76-b5a8-4009-b878-1ed15193d2e4\") "
Oct 30 19:15:33.963320 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:33.963297    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8521fe76-b5a8-4009-b878-1ed15193d2e4-config-file" pod "8521fe76-b5a8-4009-b878-1ed15193d2e4" (UID: "8521fe76-b5a8-4009-b878-1ed15193d2e4"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:15:33.965035 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:33.965014    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8521fe76-b5a8-4009-b878-1ed15193d2e4-kube-api-access-dgwwc" pod "8521fe76-b5a8-4009-b878-1ed15193d2e4" (UID: "8521fe76-b5a8-4009-b878-1ed15193d2e4"). InnerVolumeSpecName "kube-api-access-dgwwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:15:34.064139 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:34.064111    4524 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8521fe76-b5a8-4009-b878-1ed15193d2e4-config-file\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:34.064139 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:34.064134    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgwwc\" (UniqueName: \"kubernetes.io/projected/8521fe76-b5a8-4009-b878-1ed15193d2e4-kube-api-access-dgwwc\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:15:34.775486 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:34.775410    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-g74sq"
Oct 30 19:15:34.775486 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:34.775433    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-g74sq" event={"ID":"8521fe76-b5a8-4009-b878-1ed15193d2e4","Type":"ContainerDied","Data":"23bd61269b66e5c10b456fcb3ae97d2538f2c03c7562bc419485b31ea884b3e0"}
Oct 30 19:15:34.775979 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:34.775520    4524 scope.go:122] "RemoveContainer" containerID="5816ddc8eefbddc8c6c23d54b6ecbdbded4aca4de3f42d908f9ee3f12e7b23c8"
Oct 30 19:15:34.791626 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:34.791605    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-g74sq"]
Oct 30 19:15:34.795354 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:34.795334    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-g74sq"]
Oct 30 19:15:36.388037 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:36.388006    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8521fe76-b5a8-4009-b878-1ed15193d2e4" path="/var/lib/kubelet/pods/8521fe76-b5a8-4009-b878-1ed15193d2e4/volumes"
Oct 30 19:15:36.672686 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:36.672616    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-wv55z"]
Oct 30 19:15:36.672958 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:36.672945    4524 cpu_manager.go:414] "containerMap: removing container" podUID="8521fe76-b5a8-4009-b878-1ed15193d2e4" containerName="limitador"
Oct 30 19:15:36.673003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:36.672959    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8521fe76-b5a8-4009-b878-1ed15193d2e4" containerName="limitador"
Oct 30 19:15:36.673052 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:36.673043    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="8521fe76-b5a8-4009-b878-1ed15193d2e4" containerName="limitador"
Oct 30 19:15:36.677325 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:36.677294    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wv55z"
Oct 30 19:15:36.686667 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:36.686638    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-wv55z"]
Oct 30 19:15:36.692335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:36.692251    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 19:15:36.692335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:36.692307    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 30 19:15:36.692628 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:36.692468    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 19:15:36.692868 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:36.692849    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-g272x\""
Oct 30 19:15:36.785238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:36.785209    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5d311a0e-284d-4ae2-b37d-0c0542b916fc-data\") pod \"postgres-7cdbf6dc44-wv55z\" (UID: \"5d311a0e-284d-4ae2-b37d-0c0542b916fc\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wv55z"
Oct 30 19:15:36.785403 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:36.785263    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67fxw\" (UniqueName: \"kubernetes.io/projected/5d311a0e-284d-4ae2-b37d-0c0542b916fc-kube-api-access-67fxw\") pod \"postgres-7cdbf6dc44-wv55z\" (UID: \"5d311a0e-284d-4ae2-b37d-0c0542b916fc\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wv55z"
Oct 30 19:15:36.885957 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:36.885927    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67fxw\" (UniqueName: \"kubernetes.io/projected/5d311a0e-284d-4ae2-b37d-0c0542b916fc-kube-api-access-67fxw\") pod \"postgres-7cdbf6dc44-wv55z\" (UID: \"5d311a0e-284d-4ae2-b37d-0c0542b916fc\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wv55z"
Oct 30 19:15:36.886099 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:36.885993    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5d311a0e-284d-4ae2-b37d-0c0542b916fc-data\") pod \"postgres-7cdbf6dc44-wv55z\" (UID: \"5d311a0e-284d-4ae2-b37d-0c0542b916fc\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wv55z"
Oct 30 19:15:36.886423 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:36.886405    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5d311a0e-284d-4ae2-b37d-0c0542b916fc-data\") pod \"postgres-7cdbf6dc44-wv55z\" (UID: \"5d311a0e-284d-4ae2-b37d-0c0542b916fc\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wv55z"
Oct 30 19:15:36.898503 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:36.898471    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67fxw\" (UniqueName: \"kubernetes.io/projected/5d311a0e-284d-4ae2-b37d-0c0542b916fc-kube-api-access-67fxw\") pod \"postgres-7cdbf6dc44-wv55z\" (UID: \"5d311a0e-284d-4ae2-b37d-0c0542b916fc\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wv55z"
Oct 30 19:15:36.992654 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:36.992564    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wv55z"
Oct 30 19:15:37.266506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:37.266421    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-wv55z"]
Oct 30 19:15:37.269887 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:15:37.269851    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d311a0e_284d_4ae2_b37d_0c0542b916fc.slice/crio-f3f3426a6514fa07c7d16ccccbb8cdc37832162c15b614567c12a88d877d1a3a WatchSource:0}: Error finding container f3f3426a6514fa07c7d16ccccbb8cdc37832162c15b614567c12a88d877d1a3a: Status 404 returned error can't find the container with id f3f3426a6514fa07c7d16ccccbb8cdc37832162c15b614567c12a88d877d1a3a
Oct 30 19:15:37.792402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:37.792375    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wv55z" event={"ID":"5d311a0e-284d-4ae2-b37d-0c0542b916fc","Type":"ContainerStarted","Data":"f3f3426a6514fa07c7d16ccccbb8cdc37832162c15b614567c12a88d877d1a3a"}
Oct 30 19:15:44.830133 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:44.830098    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wv55z" event={"ID":"5d311a0e-284d-4ae2-b37d-0c0542b916fc","Type":"ContainerStarted","Data":"b7f9610bf3428794ea9d6f3d5a5690d730d19a848bd444feb41b2b7135d66354"}
Oct 30 19:15:44.830543 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:44.830162    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wv55z"
Oct 30 19:15:44.847791 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:44.847737    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wv55z" podStartSLOduration=2.27346889 podStartE2EDuration="8.847725163s" totalImagesPullingTime="6.574256273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:44.846998387 +0000 UTC m=+576.587989459" watchObservedRunningTime="2026-10-30 19:15:44.847725163 +0000 UTC m=+576.588716271"
Oct 30 19:15:45.624993 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:15:45.624949    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:15:50.864106 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:15:50.864041    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-wv55z"
Oct 30 19:16:01.220687 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.220648    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-577f596567-pzmff"]
Oct 30 19:16:01.235811 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.235782    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-pzmff"
Oct 30 19:16:01.237243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.237213    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-577f596567-pzmff"]
Oct 30 19:16:01.240395 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.240370    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 19:16:01.240505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.240398    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 19:16:01.240505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.240475    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dx5v7\""
Oct 30 19:16:01.240603 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.240513    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 19:16:01.378338 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.378307    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvz7m\" (UniqueName: \"kubernetes.io/projected/ff151330-22c1-492f-bfc0-81c4823586d1-kube-api-access-jvz7m\") pod \"maas-controller-577f596567-pzmff\" (UID: \"ff151330-22c1-492f-bfc0-81c4823586d1\") " pod="opendatahub/maas-controller-577f596567-pzmff"
Oct 30 19:16:01.378522 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.378362    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff151330-22c1-492f-bfc0-81c4823586d1-metrics-certs\") pod \"maas-controller-577f596567-pzmff\" (UID: \"ff151330-22c1-492f-bfc0-81c4823586d1\") " pod="opendatahub/maas-controller-577f596567-pzmff"
Oct 30 19:16:01.378522 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.378484    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff151330-22c1-492f-bfc0-81c4823586d1-webhook-cert\") pod \"maas-controller-577f596567-pzmff\" (UID: \"ff151330-22c1-492f-bfc0-81c4823586d1\") " pod="opendatahub/maas-controller-577f596567-pzmff"
Oct 30 19:16:01.479755 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.479667    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvz7m\" (UniqueName: \"kubernetes.io/projected/ff151330-22c1-492f-bfc0-81c4823586d1-kube-api-access-jvz7m\") pod \"maas-controller-577f596567-pzmff\" (UID: \"ff151330-22c1-492f-bfc0-81c4823586d1\") " pod="opendatahub/maas-controller-577f596567-pzmff"
Oct 30 19:16:01.479755 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.479713    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff151330-22c1-492f-bfc0-81c4823586d1-metrics-certs\") pod \"maas-controller-577f596567-pzmff\" (UID: \"ff151330-22c1-492f-bfc0-81c4823586d1\") " pod="opendatahub/maas-controller-577f596567-pzmff"
Oct 30 19:16:01.479755 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.479743    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff151330-22c1-492f-bfc0-81c4823586d1-webhook-cert\") pod \"maas-controller-577f596567-pzmff\" (UID: \"ff151330-22c1-492f-bfc0-81c4823586d1\") " pod="opendatahub/maas-controller-577f596567-pzmff"
Oct 30 19:16:01.482288 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.482263    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff151330-22c1-492f-bfc0-81c4823586d1-webhook-cert\") pod \"maas-controller-577f596567-pzmff\" (UID: \"ff151330-22c1-492f-bfc0-81c4823586d1\") " pod="opendatahub/maas-controller-577f596567-pzmff"
Oct 30 19:16:01.482435 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.482305    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff151330-22c1-492f-bfc0-81c4823586d1-metrics-certs\") pod \"maas-controller-577f596567-pzmff\" (UID: \"ff151330-22c1-492f-bfc0-81c4823586d1\") " pod="opendatahub/maas-controller-577f596567-pzmff"
Oct 30 19:16:01.492634 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.492611    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvz7m\" (UniqueName: \"kubernetes.io/projected/ff151330-22c1-492f-bfc0-81c4823586d1-kube-api-access-jvz7m\") pod \"maas-controller-577f596567-pzmff\" (UID: \"ff151330-22c1-492f-bfc0-81c4823586d1\") " pod="opendatahub/maas-controller-577f596567-pzmff"
Oct 30 19:16:01.549669 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.549632    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-pzmff"
Oct 30 19:16:01.605912 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.605877    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79698dcf56-jbqjp"]
Oct 30 19:16:01.609873 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.609854    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79698dcf56-jbqjp"
Oct 30 19:16:01.623797 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.623714    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79698dcf56-jbqjp"]
Oct 30 19:16:01.782391 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.782306    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-webhook-cert\") pod \"maas-controller-79698dcf56-jbqjp\" (UID: \"5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2\") " pod="opendatahub/maas-controller-79698dcf56-jbqjp"
Oct 30 19:16:01.782391 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.782346    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4gwn\" (UniqueName: \"kubernetes.io/projected/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-kube-api-access-w4gwn\") pod \"maas-controller-79698dcf56-jbqjp\" (UID: \"5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2\") " pod="opendatahub/maas-controller-79698dcf56-jbqjp"
Oct 30 19:16:01.782575 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.782461    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-metrics-certs\") pod \"maas-controller-79698dcf56-jbqjp\" (UID: \"5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2\") " pod="opendatahub/maas-controller-79698dcf56-jbqjp"
Oct 30 19:16:01.826326 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.826291    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-577f596567-pzmff"]
Oct 30 19:16:01.829042 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:16:01.829013    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff151330_22c1_492f_bfc0_81c4823586d1.slice/crio-2e2b9799fa918b7a09dc00432919060363728ba293d57e4acdd7f59ae208b3ba WatchSource:0}: Error finding container 2e2b9799fa918b7a09dc00432919060363728ba293d57e4acdd7f59ae208b3ba: Status 404 returned error can't find the container with id 2e2b9799fa918b7a09dc00432919060363728ba293d57e4acdd7f59ae208b3ba
Oct 30 19:16:01.883345 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.883310    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-metrics-certs\") pod \"maas-controller-79698dcf56-jbqjp\" (UID: \"5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2\") " pod="opendatahub/maas-controller-79698dcf56-jbqjp"
Oct 30 19:16:01.883535 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.883367    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-webhook-cert\") pod \"maas-controller-79698dcf56-jbqjp\" (UID: \"5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2\") " pod="opendatahub/maas-controller-79698dcf56-jbqjp"
Oct 30 19:16:01.883535 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.883388    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4gwn\" (UniqueName: \"kubernetes.io/projected/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-kube-api-access-w4gwn\") pod \"maas-controller-79698dcf56-jbqjp\" (UID: \"5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2\") " pod="opendatahub/maas-controller-79698dcf56-jbqjp"
Oct 30 19:16:01.885770 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.885745    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-metrics-certs\") pod \"maas-controller-79698dcf56-jbqjp\" (UID: \"5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2\") " pod="opendatahub/maas-controller-79698dcf56-jbqjp"
Oct 30 19:16:01.885884 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.885818    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-webhook-cert\") pod \"maas-controller-79698dcf56-jbqjp\" (UID: \"5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2\") " pod="opendatahub/maas-controller-79698dcf56-jbqjp"
Oct 30 19:16:01.895247 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.895222    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4gwn\" (UniqueName: \"kubernetes.io/projected/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-kube-api-access-w4gwn\") pod \"maas-controller-79698dcf56-jbqjp\" (UID: \"5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2\") " pod="opendatahub/maas-controller-79698dcf56-jbqjp"
Oct 30 19:16:01.916296 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.916262    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-pzmff" event={"ID":"ff151330-22c1-492f-bfc0-81c4823586d1","Type":"ContainerStarted","Data":"2e2b9799fa918b7a09dc00432919060363728ba293d57e4acdd7f59ae208b3ba"}
Oct 30 19:16:01.926550 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:01.926529    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79698dcf56-jbqjp"
Oct 30 19:16:02.200133 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:02.200108    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79698dcf56-jbqjp"]
Oct 30 19:16:02.923665 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:02.923586    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79698dcf56-jbqjp" event={"ID":"5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2","Type":"ContainerStarted","Data":"522f4c1dfaab8e6bd3694ff5dfd8300b55c30ea736f387b9d502bd67bbeedebf"}
Oct 30 19:16:05.944385 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:05.944347    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79698dcf56-jbqjp" event={"ID":"5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2","Type":"ContainerStarted","Data":"ccb841f7189b75406d24d1271f0b6d53b5dc026b239af59d4dc3b37ac1fbe04e"}
Oct 30 19:16:05.944855 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:05.944441    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79698dcf56-jbqjp"
Oct 30 19:16:05.945819 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:05.945790    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-pzmff" event={"ID":"ff151330-22c1-492f-bfc0-81c4823586d1","Type":"ContainerStarted","Data":"9787c0dd49dba530ab5b4452b8fa4c9e4de288db93c7bf23e329f9c8cfe3b403"}
Oct 30 19:16:05.945939 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:05.945925    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-577f596567-pzmff"
Oct 30 19:16:05.965184 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:05.965125    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-79698dcf56-jbqjp" podStartSLOduration=2.155758231 podStartE2EDuration="4.965113433s" totalImagesPullingTime="2.809355202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:16:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:16:05.963882764 +0000 UTC m=+597.704873840" watchObservedRunningTime="2026-10-30 19:16:05.965113433 +0000 UTC m=+597.706104511"
Oct 30 19:16:05.984543 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:05.984486    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-577f596567-pzmff" podStartSLOduration=1.7902131159999999 podStartE2EDuration="4.984473682s" totalImagesPullingTime="3.194260566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:16:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:16:05.982461219 +0000 UTC m=+597.723452338" watchObservedRunningTime="2026-10-30 19:16:05.984473682 +0000 UTC m=+597.725464775"
Oct 30 19:16:08.191200 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.191141    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c"]
Oct 30 19:16:08.195317 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.195296    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c"
Oct 30 19:16:08.203019 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.202965    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-7rff9\""
Oct 30 19:16:08.205195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.205145    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Oct 30 19:16:08.205846 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.205827    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Oct 30 19:16:08.206453 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.206395    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Oct 30 19:16:08.206609 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.206513    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c"]
Oct 30 19:16:08.338346 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.338311    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/7be79571-1994-41af-a1e7-63ce281f44a2-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-zsd4c\" (UID: \"7be79571-1994-41af-a1e7-63ce281f44a2\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c"
Oct 30 19:16:08.338508 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.338361    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7be79571-1994-41af-a1e7-63ce281f44a2-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-zsd4c\" (UID: \"7be79571-1994-41af-a1e7-63ce281f44a2\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c"
Oct 30 19:16:08.338562 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.338494    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llq96\" (UniqueName: \"kubernetes.io/projected/7be79571-1994-41af-a1e7-63ce281f44a2-kube-api-access-llq96\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-zsd4c\" (UID: \"7be79571-1994-41af-a1e7-63ce281f44a2\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c"
Oct 30 19:16:08.338562 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.338529    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7be79571-1994-41af-a1e7-63ce281f44a2-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-zsd4c\" (UID: \"7be79571-1994-41af-a1e7-63ce281f44a2\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c"
Oct 30 19:16:08.439345 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.439310    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llq96\" (UniqueName: \"kubernetes.io/projected/7be79571-1994-41af-a1e7-63ce281f44a2-kube-api-access-llq96\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-zsd4c\" (UID: \"7be79571-1994-41af-a1e7-63ce281f44a2\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c"
Oct 30 19:16:08.439542 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.439522    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7be79571-1994-41af-a1e7-63ce281f44a2-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-zsd4c\" (UID: \"7be79571-1994-41af-a1e7-63ce281f44a2\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c"
Oct 30 19:16:08.439611 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.439571    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/7be79571-1994-41af-a1e7-63ce281f44a2-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-zsd4c\" (UID: \"7be79571-1994-41af-a1e7-63ce281f44a2\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c"
Oct 30 19:16:08.439666 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.439625    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7be79571-1994-41af-a1e7-63ce281f44a2-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-zsd4c\" (UID: \"7be79571-1994-41af-a1e7-63ce281f44a2\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c"
Oct 30 19:16:08.441937 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.441881    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Oct 30 19:16:08.442618 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.442598    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Oct 30 19:16:08.442726 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.442652    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Oct 30 19:16:08.450140 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.450119    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7be79571-1994-41af-a1e7-63ce281f44a2-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-zsd4c\" (UID: \"7be79571-1994-41af-a1e7-63ce281f44a2\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c"
Oct 30 19:16:08.453230 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.451524    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/7be79571-1994-41af-a1e7-63ce281f44a2-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-zsd4c\" (UID: \"7be79571-1994-41af-a1e7-63ce281f44a2\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c"
Oct 30 19:16:08.459104 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.459080    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-llq96\" (UniqueName: \"kubernetes.io/projected/7be79571-1994-41af-a1e7-63ce281f44a2-kube-api-access-llq96\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-zsd4c\" (UID: \"7be79571-1994-41af-a1e7-63ce281f44a2\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c"
Oct 30 19:16:08.461642 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.461620    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7be79571-1994-41af-a1e7-63ce281f44a2-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-zsd4c\" (UID: \"7be79571-1994-41af-a1e7-63ce281f44a2\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c"
Oct 30 19:16:08.516298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.516269    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-7rff9\""
Oct 30 19:16:08.524357 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.524336    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c"
Oct 30 19:16:08.803336 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.803310    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c"]
Oct 30 19:16:08.805131 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:16:08.805110    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be79571_1994_41af_a1e7_63ce281f44a2.slice/crio-8adc0045600fccf7e5c4a8558006ffe73798ab081f306d9d68ae0495eac1d2d2 WatchSource:0}: Error finding container 8adc0045600fccf7e5c4a8558006ffe73798ab081f306d9d68ae0495eac1d2d2: Status 404 returned error can't find the container with id 8adc0045600fccf7e5c4a8558006ffe73798ab081f306d9d68ae0495eac1d2d2
Oct 30 19:16:08.965410 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:08.965378    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c" event={"ID":"7be79571-1994-41af-a1e7-63ce281f44a2","Type":"ContainerStarted","Data":"8adc0045600fccf7e5c4a8558006ffe73798ab081f306d9d68ae0495eac1d2d2"}
Oct 30 19:16:15.629911 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:16:15.629866    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:16:16.953491 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:16.953457    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79698dcf56-jbqjp"
Oct 30 19:16:16.953980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:16.953541    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-577f596567-pzmff"
Oct 30 19:16:17.031401 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:17.031368    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-577f596567-pzmff"]
Oct 30 19:16:17.031591 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:17.031559    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-577f596567-pzmff" podUID="ff151330-22c1-492f-bfc0-81c4823586d1" containerName="manager" containerID="cri-o://9787c0dd49dba530ab5b4452b8fa4c9e4de288db93c7bf23e329f9c8cfe3b403" gracePeriod=30
Oct 30 19:16:17.283590 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:17.283566    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-pzmff"
Oct 30 19:16:17.318433 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:17.318405    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff151330-22c1-492f-bfc0-81c4823586d1-kube-api-access-jvz7m\" (UniqueName: \"kubernetes.io/projected/ff151330-22c1-492f-bfc0-81c4823586d1-kube-api-access-jvz7m\") pod \"ff151330-22c1-492f-bfc0-81c4823586d1\" (UID: \"ff151330-22c1-492f-bfc0-81c4823586d1\") "
Oct 30 19:16:17.318580 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:17.318484    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ff151330-22c1-492f-bfc0-81c4823586d1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff151330-22c1-492f-bfc0-81c4823586d1-metrics-certs\") pod \"ff151330-22c1-492f-bfc0-81c4823586d1\" (UID: \"ff151330-22c1-492f-bfc0-81c4823586d1\") "
Oct 30 19:16:17.318580 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:17.318512    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ff151330-22c1-492f-bfc0-81c4823586d1-webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff151330-22c1-492f-bfc0-81c4823586d1-webhook-cert\") pod \"ff151330-22c1-492f-bfc0-81c4823586d1\" (UID: \"ff151330-22c1-492f-bfc0-81c4823586d1\") "
Oct 30 19:16:17.323582 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:17.323545    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff151330-22c1-492f-bfc0-81c4823586d1-webhook-cert" pod "ff151330-22c1-492f-bfc0-81c4823586d1" (UID: "ff151330-22c1-492f-bfc0-81c4823586d1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:17.323709 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:17.323626    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff151330-22c1-492f-bfc0-81c4823586d1-metrics-certs" pod "ff151330-22c1-492f-bfc0-81c4823586d1" (UID: "ff151330-22c1-492f-bfc0-81c4823586d1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:17.323783 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:17.323727    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff151330-22c1-492f-bfc0-81c4823586d1-kube-api-access-jvz7m" pod "ff151330-22c1-492f-bfc0-81c4823586d1" (UID: "ff151330-22c1-492f-bfc0-81c4823586d1"). InnerVolumeSpecName "kube-api-access-jvz7m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:17.419414 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:17.419384    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ff151330-22c1-492f-bfc0-81c4823586d1-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:17.419414 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:17.419412    4524 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ff151330-22c1-492f-bfc0-81c4823586d1-webhook-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:17.419664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:17.419426    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvz7m\" (UniqueName: \"kubernetes.io/projected/ff151330-22c1-492f-bfc0-81c4823586d1-kube-api-access-jvz7m\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:18.019578 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:18.019539    4524 generic.go:356] "Generic (PLEG): container finished" podID="ff151330-22c1-492f-bfc0-81c4823586d1" containerID="9787c0dd49dba530ab5b4452b8fa4c9e4de288db93c7bf23e329f9c8cfe3b403" exitCode=0
Oct 30 19:16:18.020065 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:18.019581    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-pzmff"
Oct 30 19:16:18.020065 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:18.019624    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-pzmff" event={"ID":"ff151330-22c1-492f-bfc0-81c4823586d1","Type":"ContainerDied","Data":"9787c0dd49dba530ab5b4452b8fa4c9e4de288db93c7bf23e329f9c8cfe3b403"}
Oct 30 19:16:18.020065 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:18.019680    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-pzmff" event={"ID":"ff151330-22c1-492f-bfc0-81c4823586d1","Type":"ContainerDied","Data":"2e2b9799fa918b7a09dc00432919060363728ba293d57e4acdd7f59ae208b3ba"}
Oct 30 19:16:18.020065 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:18.019713    4524 scope.go:122] "RemoveContainer" containerID="9787c0dd49dba530ab5b4452b8fa4c9e4de288db93c7bf23e329f9c8cfe3b403"
Oct 30 19:16:18.029820 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:18.029805    4524 scope.go:122] "RemoveContainer" containerID="9787c0dd49dba530ab5b4452b8fa4c9e4de288db93c7bf23e329f9c8cfe3b403"
Oct 30 19:16:18.030142 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:16:18.030114    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9787c0dd49dba530ab5b4452b8fa4c9e4de288db93c7bf23e329f9c8cfe3b403\": container with ID starting with 9787c0dd49dba530ab5b4452b8fa4c9e4de288db93c7bf23e329f9c8cfe3b403 not found: ID does not exist" containerID="9787c0dd49dba530ab5b4452b8fa4c9e4de288db93c7bf23e329f9c8cfe3b403"
Oct 30 19:16:18.030264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:18.030150    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9787c0dd49dba530ab5b4452b8fa4c9e4de288db93c7bf23e329f9c8cfe3b403"} err="failed to get container status \"9787c0dd49dba530ab5b4452b8fa4c9e4de288db93c7bf23e329f9c8cfe3b403\": rpc error: code = NotFound desc = could not find container \"9787c0dd49dba530ab5b4452b8fa4c9e4de288db93c7bf23e329f9c8cfe3b403\": container with ID starting with 9787c0dd49dba530ab5b4452b8fa4c9e4de288db93c7bf23e329f9c8cfe3b403 not found: ID does not exist"
Oct 30 19:16:18.043519 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:18.043496    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-577f596567-pzmff"]
Oct 30 19:16:18.047646 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:18.047620    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-577f596567-pzmff"]
Oct 30 19:16:18.388606 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:18.388577    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ff151330-22c1-492f-bfc0-81c4823586d1" path="/var/lib/kubelet/pods/ff151330-22c1-492f-bfc0-81c4823586d1/volumes"
Oct 30 19:16:20.034295 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:20.034220    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c" event={"ID":"7be79571-1994-41af-a1e7-63ce281f44a2","Type":"ContainerStarted","Data":"85c45390ca4baaa7d5214c65be4de5836e2d4d9867cb7938f019de687848cf0a"}
Oct 30 19:16:20.034295 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:20.034287    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c"
Oct 30 19:16:20.053385 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:20.053328    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c" podStartSLOduration=1.079977198 podStartE2EDuration="12.053310059s" totalImagesPullingTime="10.973332861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:16:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:16:20.052439224 +0000 UTC m=+611.793430302" watchObservedRunningTime="2026-10-30 19:16:20.053310059 +0000 UTC m=+611.794301138"
Oct 30 19:16:31.044928 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:31.044891    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-zsd4c"
Oct 30 19:16:36.314373 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.314337    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"]
Oct 30 19:16:36.314847 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.314709    4524 cpu_manager.go:414] "containerMap: removing container" podUID="ff151330-22c1-492f-bfc0-81c4823586d1" containerName="manager"
Oct 30 19:16:36.314847 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.314720    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff151330-22c1-492f-bfc0-81c4823586d1" containerName="manager"
Oct 30 19:16:36.314847 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.314775    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff151330-22c1-492f-bfc0-81c4823586d1" containerName="manager"
Oct 30 19:16:36.318045 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.318025    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"
Oct 30 19:16:36.322834 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.322810    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 19:16:36.322834 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.322816    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-f8msk\""
Oct 30 19:16:36.323047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.323011    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 19:16:36.328569 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.328546    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"]
Oct 30 19:16:36.369082 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.369045    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wchf\" (UniqueName: \"kubernetes.io/projected/502e23a9-33af-4bdc-88fb-21f189d8ec62-kube-api-access-8wchf\") pod \"maas-api-5c48877c75-5qltt\" (UID: \"502e23a9-33af-4bdc-88fb-21f189d8ec62\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"
Oct 30 19:16:36.369255 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.369121    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/502e23a9-33af-4bdc-88fb-21f189d8ec62-metrics-certs\") pod \"maas-api-5c48877c75-5qltt\" (UID: \"502e23a9-33af-4bdc-88fb-21f189d8ec62\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"
Oct 30 19:16:36.369255 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.369231    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/502e23a9-33af-4bdc-88fb-21f189d8ec62-openshift-service-ca\") pod \"maas-api-5c48877c75-5qltt\" (UID: \"502e23a9-33af-4bdc-88fb-21f189d8ec62\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"
Oct 30 19:16:36.369381 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.369283    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/502e23a9-33af-4bdc-88fb-21f189d8ec62-maas-api-tls\") pod \"maas-api-5c48877c75-5qltt\" (UID: \"502e23a9-33af-4bdc-88fb-21f189d8ec62\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"
Oct 30 19:16:36.469878 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.469846    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/502e23a9-33af-4bdc-88fb-21f189d8ec62-openshift-service-ca\") pod \"maas-api-5c48877c75-5qltt\" (UID: \"502e23a9-33af-4bdc-88fb-21f189d8ec62\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"
Oct 30 19:16:36.470042 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.469909    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/502e23a9-33af-4bdc-88fb-21f189d8ec62-maas-api-tls\") pod \"maas-api-5c48877c75-5qltt\" (UID: \"502e23a9-33af-4bdc-88fb-21f189d8ec62\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"
Oct 30 19:16:36.470123 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.470099    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wchf\" (UniqueName: \"kubernetes.io/projected/502e23a9-33af-4bdc-88fb-21f189d8ec62-kube-api-access-8wchf\") pod \"maas-api-5c48877c75-5qltt\" (UID: \"502e23a9-33af-4bdc-88fb-21f189d8ec62\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"
Oct 30 19:16:36.470212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.470198    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/502e23a9-33af-4bdc-88fb-21f189d8ec62-metrics-certs\") pod \"maas-api-5c48877c75-5qltt\" (UID: \"502e23a9-33af-4bdc-88fb-21f189d8ec62\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"
Oct 30 19:16:36.470680 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.470654    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/502e23a9-33af-4bdc-88fb-21f189d8ec62-openshift-service-ca\") pod \"maas-api-5c48877c75-5qltt\" (UID: \"502e23a9-33af-4bdc-88fb-21f189d8ec62\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"
Oct 30 19:16:36.472802 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.472776    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/502e23a9-33af-4bdc-88fb-21f189d8ec62-maas-api-tls\") pod \"maas-api-5c48877c75-5qltt\" (UID: \"502e23a9-33af-4bdc-88fb-21f189d8ec62\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"
Oct 30 19:16:36.473052 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.473029    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/502e23a9-33af-4bdc-88fb-21f189d8ec62-metrics-certs\") pod \"maas-api-5c48877c75-5qltt\" (UID: \"502e23a9-33af-4bdc-88fb-21f189d8ec62\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"
Oct 30 19:16:36.482302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.482281    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wchf\" (UniqueName: \"kubernetes.io/projected/502e23a9-33af-4bdc-88fb-21f189d8ec62-kube-api-access-8wchf\") pod \"maas-api-5c48877c75-5qltt\" (UID: \"502e23a9-33af-4bdc-88fb-21f189d8ec62\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"
Oct 30 19:16:36.634337 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.634309    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"
Oct 30 19:16:36.909671 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.909595    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"]
Oct 30 19:16:36.913464 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:16:36.913435    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod502e23a9_33af_4bdc_88fb_21f189d8ec62.slice/crio-1b52312efca0f32f54abf666c9d71c6e6611e262aeac90590eb612356a836933 WatchSource:0}: Error finding container 1b52312efca0f32f54abf666c9d71c6e6611e262aeac90590eb612356a836933: Status 404 returned error can't find the container with id 1b52312efca0f32f54abf666c9d71c6e6611e262aeac90590eb612356a836933
Oct 30 19:16:36.914846 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:36.914826    4524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:16:37.113937 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:37.113896    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt" event={"ID":"502e23a9-33af-4bdc-88fb-21f189d8ec62","Type":"ContainerStarted","Data":"1b52312efca0f32f54abf666c9d71c6e6611e262aeac90590eb612356a836933"}
Oct 30 19:16:40.131409 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:40.131368    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt" event={"ID":"502e23a9-33af-4bdc-88fb-21f189d8ec62","Type":"ContainerStarted","Data":"9b60e5ebee3861cc5a15047b102d9d3a7f9c7583f0b470c8b26e8d18cd6ef42f"}
Oct 30 19:16:40.131823 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:40.131496    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"
Oct 30 19:16:40.152623 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:40.152574    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt" podStartSLOduration=1.856681912 podStartE2EDuration="4.152560414s" totalImagesPullingTime="2.295878502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:16:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:16:40.150490943 +0000 UTC m=+631.891482023" watchObservedRunningTime="2026-10-30 19:16:40.152560414 +0000 UTC m=+631.893551493"
Oct 30 19:16:45.624042 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:16:45.624002    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:16:46.141250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:16:46.141223    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"
Oct 30 19:17:15.624130 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:17:15.624090    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:17:42.909597 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:42.909564    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"]
Oct 30 19:17:42.913251 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:42.913228    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:42.917068 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:42.917042    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 19:17:42.917215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:42.917071    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 19:17:42.917215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:42.917107    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-fvzqm\""
Oct 30 19:17:42.917291 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:42.917233    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Oct 30 19:17:42.925124 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:42.925095    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"]
Oct 30 19:17:42.977037 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:42.977005    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28cfff8e-626d-4a42-8d25-b5296c17db9c-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-vc5q8\" (UID: \"28cfff8e-626d-4a42-8d25-b5296c17db9c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:42.977037 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:42.977037    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28cfff8e-626d-4a42-8d25-b5296c17db9c-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-vc5q8\" (UID: \"28cfff8e-626d-4a42-8d25-b5296c17db9c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:42.977273 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:42.977060    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx86d\" (UniqueName: \"kubernetes.io/projected/28cfff8e-626d-4a42-8d25-b5296c17db9c-kube-api-access-hx86d\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-vc5q8\" (UID: \"28cfff8e-626d-4a42-8d25-b5296c17db9c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:42.977273 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:42.977131    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28cfff8e-626d-4a42-8d25-b5296c17db9c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-vc5q8\" (UID: \"28cfff8e-626d-4a42-8d25-b5296c17db9c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:42.977273 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:42.977204    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28cfff8e-626d-4a42-8d25-b5296c17db9c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-vc5q8\" (UID: \"28cfff8e-626d-4a42-8d25-b5296c17db9c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:42.977273 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:42.977243    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28cfff8e-626d-4a42-8d25-b5296c17db9c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-vc5q8\" (UID: \"28cfff8e-626d-4a42-8d25-b5296c17db9c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:43.077805 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:43.077773    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28cfff8e-626d-4a42-8d25-b5296c17db9c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-vc5q8\" (UID: \"28cfff8e-626d-4a42-8d25-b5296c17db9c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:43.077940 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:43.077816    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28cfff8e-626d-4a42-8d25-b5296c17db9c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-vc5q8\" (UID: \"28cfff8e-626d-4a42-8d25-b5296c17db9c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:43.077940 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:43.077868    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28cfff8e-626d-4a42-8d25-b5296c17db9c-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-vc5q8\" (UID: \"28cfff8e-626d-4a42-8d25-b5296c17db9c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:43.077940 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:43.077884    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28cfff8e-626d-4a42-8d25-b5296c17db9c-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-vc5q8\" (UID: \"28cfff8e-626d-4a42-8d25-b5296c17db9c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:43.077940 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:43.077904    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx86d\" (UniqueName: \"kubernetes.io/projected/28cfff8e-626d-4a42-8d25-b5296c17db9c-kube-api-access-hx86d\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-vc5q8\" (UID: \"28cfff8e-626d-4a42-8d25-b5296c17db9c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:43.077940 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:43.077936    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28cfff8e-626d-4a42-8d25-b5296c17db9c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-vc5q8\" (UID: \"28cfff8e-626d-4a42-8d25-b5296c17db9c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:43.078372 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:43.078353    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/28cfff8e-626d-4a42-8d25-b5296c17db9c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-vc5q8\" (UID: \"28cfff8e-626d-4a42-8d25-b5296c17db9c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:43.078437 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:43.078384    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28cfff8e-626d-4a42-8d25-b5296c17db9c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-vc5q8\" (UID: \"28cfff8e-626d-4a42-8d25-b5296c17db9c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:43.078477 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:43.078439    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/28cfff8e-626d-4a42-8d25-b5296c17db9c-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-vc5q8\" (UID: \"28cfff8e-626d-4a42-8d25-b5296c17db9c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:43.080521 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:43.080492    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/28cfff8e-626d-4a42-8d25-b5296c17db9c-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-vc5q8\" (UID: \"28cfff8e-626d-4a42-8d25-b5296c17db9c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:43.080817 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:43.080799    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28cfff8e-626d-4a42-8d25-b5296c17db9c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-vc5q8\" (UID: \"28cfff8e-626d-4a42-8d25-b5296c17db9c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:43.090023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:43.090002    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx86d\" (UniqueName: \"kubernetes.io/projected/28cfff8e-626d-4a42-8d25-b5296c17db9c-kube-api-access-hx86d\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-vc5q8\" (UID: \"28cfff8e-626d-4a42-8d25-b5296c17db9c\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:43.227636 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:43.227554    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:43.504795 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:43.504768    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"]
Oct 30 19:17:43.506730 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:17:43.506695    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28cfff8e_626d_4a42_8d25_b5296c17db9c.slice/crio-8651ab7bbda162ff01fd195182bde67a7f97ba7e955ebe1aa3aa412dd8c46ba3 WatchSource:0}: Error finding container 8651ab7bbda162ff01fd195182bde67a7f97ba7e955ebe1aa3aa412dd8c46ba3: Status 404 returned error can't find the container with id 8651ab7bbda162ff01fd195182bde67a7f97ba7e955ebe1aa3aa412dd8c46ba3
Oct 30 19:17:44.061770 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.061740    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"]
Oct 30 19:17:44.065831 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.065807    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:44.078441 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.078416    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"]
Oct 30 19:17:44.079316 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.078705    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Oct 30 19:17:44.187568 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.187530    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/420a5dfa-7b17-480e-bbb7-71f7c290f24e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79\" (UID: \"420a5dfa-7b17-480e-bbb7-71f7c290f24e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:44.187763 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.187652    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/420a5dfa-7b17-480e-bbb7-71f7c290f24e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79\" (UID: \"420a5dfa-7b17-480e-bbb7-71f7c290f24e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:44.187763 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.187700    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmdcs\" (UniqueName: \"kubernetes.io/projected/420a5dfa-7b17-480e-bbb7-71f7c290f24e-kube-api-access-cmdcs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79\" (UID: \"420a5dfa-7b17-480e-bbb7-71f7c290f24e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:44.187763 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.187753    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/420a5dfa-7b17-480e-bbb7-71f7c290f24e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79\" (UID: \"420a5dfa-7b17-480e-bbb7-71f7c290f24e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:44.188016 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.187946    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/420a5dfa-7b17-480e-bbb7-71f7c290f24e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79\" (UID: \"420a5dfa-7b17-480e-bbb7-71f7c290f24e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:44.188097 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.188076    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/420a5dfa-7b17-480e-bbb7-71f7c290f24e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79\" (UID: \"420a5dfa-7b17-480e-bbb7-71f7c290f24e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:44.289304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.288586    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/420a5dfa-7b17-480e-bbb7-71f7c290f24e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79\" (UID: \"420a5dfa-7b17-480e-bbb7-71f7c290f24e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:44.289304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.288664    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/420a5dfa-7b17-480e-bbb7-71f7c290f24e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79\" (UID: \"420a5dfa-7b17-480e-bbb7-71f7c290f24e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:44.289304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.288709    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/420a5dfa-7b17-480e-bbb7-71f7c290f24e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79\" (UID: \"420a5dfa-7b17-480e-bbb7-71f7c290f24e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:44.289304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.288748    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/420a5dfa-7b17-480e-bbb7-71f7c290f24e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79\" (UID: \"420a5dfa-7b17-480e-bbb7-71f7c290f24e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:44.289304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.288776    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmdcs\" (UniqueName: \"kubernetes.io/projected/420a5dfa-7b17-480e-bbb7-71f7c290f24e-kube-api-access-cmdcs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79\" (UID: \"420a5dfa-7b17-480e-bbb7-71f7c290f24e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:44.289304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.288814    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/420a5dfa-7b17-480e-bbb7-71f7c290f24e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79\" (UID: \"420a5dfa-7b17-480e-bbb7-71f7c290f24e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:44.289304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.288948    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/420a5dfa-7b17-480e-bbb7-71f7c290f24e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79\" (UID: \"420a5dfa-7b17-480e-bbb7-71f7c290f24e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:44.289304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.289210    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/420a5dfa-7b17-480e-bbb7-71f7c290f24e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79\" (UID: \"420a5dfa-7b17-480e-bbb7-71f7c290f24e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:44.289853 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.289379    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/420a5dfa-7b17-480e-bbb7-71f7c290f24e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79\" (UID: \"420a5dfa-7b17-480e-bbb7-71f7c290f24e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:44.291834 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.291807    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/420a5dfa-7b17-480e-bbb7-71f7c290f24e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79\" (UID: \"420a5dfa-7b17-480e-bbb7-71f7c290f24e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:44.292542 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.292501    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/420a5dfa-7b17-480e-bbb7-71f7c290f24e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79\" (UID: \"420a5dfa-7b17-480e-bbb7-71f7c290f24e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:44.301675 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.301646    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmdcs\" (UniqueName: \"kubernetes.io/projected/420a5dfa-7b17-480e-bbb7-71f7c290f24e-kube-api-access-cmdcs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79\" (UID: \"420a5dfa-7b17-480e-bbb7-71f7c290f24e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:44.398048 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.397996    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:44.442367 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.442325    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8" event={"ID":"28cfff8e-626d-4a42-8d25-b5296c17db9c","Type":"ContainerStarted","Data":"8651ab7bbda162ff01fd195182bde67a7f97ba7e955ebe1aa3aa412dd8c46ba3"}
Oct 30 19:17:44.687780 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:44.687749    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"]
Oct 30 19:17:44.689019 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:17:44.688993    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod420a5dfa_7b17_480e_bbb7_71f7c290f24e.slice/crio-4ed88274a3195f694db5c4cc9cf4ceb3531dff8f505b5bf3de5ae946e065bffa WatchSource:0}: Error finding container 4ed88274a3195f694db5c4cc9cf4ceb3531dff8f505b5bf3de5ae946e065bffa: Status 404 returned error can't find the container with id 4ed88274a3195f694db5c4cc9cf4ceb3531dff8f505b5bf3de5ae946e065bffa
Oct 30 19:17:45.449210 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:45.449164    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79" event={"ID":"420a5dfa-7b17-480e-bbb7-71f7c290f24e","Type":"ContainerStarted","Data":"4ed88274a3195f694db5c4cc9cf4ceb3531dff8f505b5bf3de5ae946e065bffa"}
Oct 30 19:17:45.450781 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:45.450748    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8" event={"ID":"28cfff8e-626d-4a42-8d25-b5296c17db9c","Type":"ContainerStarted","Data":"feba79614609495f24cfdebbc2e898e61e8d3022f00afd624a9dec5b82adbdc4"}
Oct 30 19:17:45.473844 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:45.473791    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8" podStartSLOduration=1.935980931 podStartE2EDuration="3.473774307s" totalImagesPullingTime="1.537793376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:17:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:17:45.4703827 +0000 UTC m=+697.211373779" watchObservedRunningTime="2026-10-30 19:17:45.473774307 +0000 UTC m=+697.214765387"
Oct 30 19:17:45.627114 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:17:45.627077    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:17:46.228717 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:46.228684    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:46.459238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:46.459196    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79" event={"ID":"420a5dfa-7b17-480e-bbb7-71f7c290f24e","Type":"ContainerStarted","Data":"53a44d89a8d524e2c65f8febd02f3b8cd2a15305a98eea77fc634a123f522718"}
Oct 30 19:17:46.487564 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:46.487475    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79" podStartSLOduration=1.692924187 podStartE2EDuration="2.487462956s" totalImagesPullingTime="794.538769ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:17:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:17:46.485846848 +0000 UTC m=+698.226837927" watchObservedRunningTime="2026-10-30 19:17:46.487462956 +0000 UTC m=+698.228454036"
Oct 30 19:17:47.399245 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.399205    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:47.636685 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.636651    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"]
Oct 30 19:17:47.640627 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.640605    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:47.655946 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.655900    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Oct 30 19:17:47.680862 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.680832    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"]
Oct 30 19:17:47.722615 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.722580    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f53bd27-b284-4538-8b5e-24a32917541b-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n\" (UID: \"7f53bd27-b284-4538-8b5e-24a32917541b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:47.722737 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.722627    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f53bd27-b284-4538-8b5e-24a32917541b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n\" (UID: \"7f53bd27-b284-4538-8b5e-24a32917541b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:47.722784 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.722763    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f53bd27-b284-4538-8b5e-24a32917541b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n\" (UID: \"7f53bd27-b284-4538-8b5e-24a32917541b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:47.722824 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.722797    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f53bd27-b284-4538-8b5e-24a32917541b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n\" (UID: \"7f53bd27-b284-4538-8b5e-24a32917541b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:47.722857 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.722820    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzxxh\" (UniqueName: \"kubernetes.io/projected/7f53bd27-b284-4538-8b5e-24a32917541b-kube-api-access-vzxxh\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n\" (UID: \"7f53bd27-b284-4538-8b5e-24a32917541b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:47.722943 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.722929    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f53bd27-b284-4538-8b5e-24a32917541b-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n\" (UID: \"7f53bd27-b284-4538-8b5e-24a32917541b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:47.823775 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.823731    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f53bd27-b284-4538-8b5e-24a32917541b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n\" (UID: \"7f53bd27-b284-4538-8b5e-24a32917541b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:47.823942 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.823784    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f53bd27-b284-4538-8b5e-24a32917541b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n\" (UID: \"7f53bd27-b284-4538-8b5e-24a32917541b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:47.823942 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.823811    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzxxh\" (UniqueName: \"kubernetes.io/projected/7f53bd27-b284-4538-8b5e-24a32917541b-kube-api-access-vzxxh\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n\" (UID: \"7f53bd27-b284-4538-8b5e-24a32917541b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:47.823942 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.823847    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f53bd27-b284-4538-8b5e-24a32917541b-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n\" (UID: \"7f53bd27-b284-4538-8b5e-24a32917541b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:47.823942 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.823897    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f53bd27-b284-4538-8b5e-24a32917541b-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n\" (UID: \"7f53bd27-b284-4538-8b5e-24a32917541b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:47.824200 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.823944    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f53bd27-b284-4538-8b5e-24a32917541b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n\" (UID: \"7f53bd27-b284-4538-8b5e-24a32917541b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:47.824271 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.824198    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f53bd27-b284-4538-8b5e-24a32917541b-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n\" (UID: \"7f53bd27-b284-4538-8b5e-24a32917541b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:47.824329 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.824305    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f53bd27-b284-4538-8b5e-24a32917541b-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n\" (UID: \"7f53bd27-b284-4538-8b5e-24a32917541b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:47.824378 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.824348    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f53bd27-b284-4538-8b5e-24a32917541b-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n\" (UID: \"7f53bd27-b284-4538-8b5e-24a32917541b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:47.826105 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.826082    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f53bd27-b284-4538-8b5e-24a32917541b-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n\" (UID: \"7f53bd27-b284-4538-8b5e-24a32917541b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:47.826274 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.826257    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f53bd27-b284-4538-8b5e-24a32917541b-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n\" (UID: \"7f53bd27-b284-4538-8b5e-24a32917541b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:47.841300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.841277    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzxxh\" (UniqueName: \"kubernetes.io/projected/7f53bd27-b284-4538-8b5e-24a32917541b-kube-api-access-vzxxh\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n\" (UID: \"7f53bd27-b284-4538-8b5e-24a32917541b\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:47.954286 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:47.954205    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:48.231416 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:48.231383    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"]
Oct 30 19:17:48.232464 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:17:48.232437    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f53bd27_b284_4538_8b5e_24a32917541b.slice/crio-0394ed861d23590e5f276fa0d7f09ac07b0125474210d682737d3ee566b14d3d WatchSource:0}: Error finding container 0394ed861d23590e5f276fa0d7f09ac07b0125474210d682737d3ee566b14d3d: Status 404 returned error can't find the container with id 0394ed861d23590e5f276fa0d7f09ac07b0125474210d682737d3ee566b14d3d
Oct 30 19:17:48.470802 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:48.470766    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n" event={"ID":"7f53bd27-b284-4538-8b5e-24a32917541b","Type":"ContainerStarted","Data":"0394ed861d23590e5f276fa0d7f09ac07b0125474210d682737d3ee566b14d3d"}
Oct 30 19:17:49.477702 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:49.477672    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n" event={"ID":"7f53bd27-b284-4538-8b5e-24a32917541b","Type":"ContainerStarted","Data":"8903496c283e6eb6d1111dacf08f507d3d8ab49ebf85bef1e94b3001ab0369b3"}
Oct 30 19:17:49.501132 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:49.501076    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n" podStartSLOduration=2.170197969 podStartE2EDuration="2.501057928s" totalImagesPullingTime="330.859959ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:17:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:17:49.498852228 +0000 UTC m=+701.239843309" watchObservedRunningTime="2026-10-30 19:17:49.501057928 +0000 UTC m=+701.242049008"
Oct 30 19:17:49.955390 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:49.955347    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:50.485365 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.485333    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"]
Oct 30 19:17:50.489288 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.489258    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:50.502275 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.500623    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"]
Oct 30 19:17:50.515328 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.511055    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Oct 30 19:17:50.548408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.548381    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d44ab249-0b8f-4f57-ae7f-b6071a229e01-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zgk94\" (UID: \"d44ab249-0b8f-4f57-ae7f-b6071a229e01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:50.548544 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.548413    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d44ab249-0b8f-4f57-ae7f-b6071a229e01-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zgk94\" (UID: \"d44ab249-0b8f-4f57-ae7f-b6071a229e01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:50.548544 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.548429    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d44ab249-0b8f-4f57-ae7f-b6071a229e01-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zgk94\" (UID: \"d44ab249-0b8f-4f57-ae7f-b6071a229e01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:50.548661 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.548572    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d44ab249-0b8f-4f57-ae7f-b6071a229e01-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zgk94\" (UID: \"d44ab249-0b8f-4f57-ae7f-b6071a229e01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:50.548661 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.548616    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47zrn\" (UniqueName: \"kubernetes.io/projected/d44ab249-0b8f-4f57-ae7f-b6071a229e01-kube-api-access-47zrn\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zgk94\" (UID: \"d44ab249-0b8f-4f57-ae7f-b6071a229e01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:50.548768 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.548744    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d44ab249-0b8f-4f57-ae7f-b6071a229e01-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zgk94\" (UID: \"d44ab249-0b8f-4f57-ae7f-b6071a229e01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:50.649695 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.649662    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d44ab249-0b8f-4f57-ae7f-b6071a229e01-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zgk94\" (UID: \"d44ab249-0b8f-4f57-ae7f-b6071a229e01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:50.649856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.649717    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d44ab249-0b8f-4f57-ae7f-b6071a229e01-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zgk94\" (UID: \"d44ab249-0b8f-4f57-ae7f-b6071a229e01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:50.649856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.649733    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d44ab249-0b8f-4f57-ae7f-b6071a229e01-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zgk94\" (UID: \"d44ab249-0b8f-4f57-ae7f-b6071a229e01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:50.649963 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.649863    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d44ab249-0b8f-4f57-ae7f-b6071a229e01-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zgk94\" (UID: \"d44ab249-0b8f-4f57-ae7f-b6071a229e01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:50.650020 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.649982    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d44ab249-0b8f-4f57-ae7f-b6071a229e01-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zgk94\" (UID: \"d44ab249-0b8f-4f57-ae7f-b6071a229e01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:50.650076 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.650016    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47zrn\" (UniqueName: \"kubernetes.io/projected/d44ab249-0b8f-4f57-ae7f-b6071a229e01-kube-api-access-47zrn\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zgk94\" (UID: \"d44ab249-0b8f-4f57-ae7f-b6071a229e01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:50.650286 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.650262    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d44ab249-0b8f-4f57-ae7f-b6071a229e01-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zgk94\" (UID: \"d44ab249-0b8f-4f57-ae7f-b6071a229e01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:50.650407 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.650341    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d44ab249-0b8f-4f57-ae7f-b6071a229e01-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zgk94\" (UID: \"d44ab249-0b8f-4f57-ae7f-b6071a229e01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:50.650476 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.650392    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d44ab249-0b8f-4f57-ae7f-b6071a229e01-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zgk94\" (UID: \"d44ab249-0b8f-4f57-ae7f-b6071a229e01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:50.651995 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.651971    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d44ab249-0b8f-4f57-ae7f-b6071a229e01-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zgk94\" (UID: \"d44ab249-0b8f-4f57-ae7f-b6071a229e01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:50.652322 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.652304    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d44ab249-0b8f-4f57-ae7f-b6071a229e01-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zgk94\" (UID: \"d44ab249-0b8f-4f57-ae7f-b6071a229e01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:50.662335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.662314    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47zrn\" (UniqueName: \"kubernetes.io/projected/d44ab249-0b8f-4f57-ae7f-b6071a229e01-kube-api-access-47zrn\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-zgk94\" (UID: \"d44ab249-0b8f-4f57-ae7f-b6071a229e01\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:50.815039 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:50.814956    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:51.094592 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:51.094569    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"]
Oct 30 19:17:51.096533 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:17:51.096503    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd44ab249_0b8f_4f57_ae7f_b6071a229e01.slice/crio-694742ea20965c5745cf11160b4e234adc133b0428396d4003ab96f29b411d0e WatchSource:0}: Error finding container 694742ea20965c5745cf11160b4e234adc133b0428396d4003ab96f29b411d0e: Status 404 returned error can't find the container with id 694742ea20965c5745cf11160b4e234adc133b0428396d4003ab96f29b411d0e
Oct 30 19:17:51.492109 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:51.492079    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94" event={"ID":"d44ab249-0b8f-4f57-ae7f-b6071a229e01","Type":"ContainerStarted","Data":"694742ea20965c5745cf11160b4e234adc133b0428396d4003ab96f29b411d0e"}
Oct 30 19:17:52.499264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:52.499231    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94" event={"ID":"d44ab249-0b8f-4f57-ae7f-b6071a229e01","Type":"ContainerStarted","Data":"53438b840f3d3c8f26485fc508651669d2d67b42598cc3592e1edd7c4af70e11"}
Oct 30 19:17:52.523355 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:52.523309    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94" podStartSLOduration=2.205096585 podStartE2EDuration="2.52329347s" totalImagesPullingTime="318.196885ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:17:52.520869308 +0000 UTC m=+704.261860387" watchObservedRunningTime="2026-10-30 19:17:52.52329347 +0000 UTC m=+704.264284548"
Oct 30 19:17:52.815555 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:52.815474    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:17:53.228474 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:53.228438    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:53.241002 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:53.240980    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:53.516436 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:53.516358    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-vc5q8"
Oct 30 19:17:54.398604 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:54.398568    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:54.410923 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:54.410894    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:54.523207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:54.523180    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79"
Oct 30 19:17:57.954459 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:57.954424    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:57.966384 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:57.966364    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:17:58.541689 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:17:58.541664    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n"
Oct 30 19:18:00.815645 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:00.815566    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:18:00.827652 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:00.827629    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:18:01.557437 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:01.557402    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-zgk94"
Oct 30 19:18:05.643126 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:05.643088    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn"]
Oct 30 19:18:05.649809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:05.649787    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn"
Oct 30 19:18:05.654540 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:05.654516    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Oct 30 19:18:05.654678 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:05.654622    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-cnjvx\""
Oct 30 19:18:05.654678 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:05.654662    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Oct 30 19:18:05.657327 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:05.657268    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn"]
Oct 30 19:18:05.781900 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:05.781867    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkq8j\" (UniqueName: \"kubernetes.io/projected/39bfb8a2-4ab0-478c-aa24-376a86fbdec4-kube-api-access-dkq8j\") pod \"ai-gateway-controller-5d9cd99f45-w9zmn\" (UID: \"39bfb8a2-4ab0-478c-aa24-376a86fbdec4\") " pod="opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn"
Oct 30 19:18:05.782066 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:05.781927    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39bfb8a2-4ab0-478c-aa24-376a86fbdec4-metrics-certs\") pod \"ai-gateway-controller-5d9cd99f45-w9zmn\" (UID: \"39bfb8a2-4ab0-478c-aa24-376a86fbdec4\") " pod="opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn"
Oct 30 19:18:05.883405 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:05.883377    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkq8j\" (UniqueName: \"kubernetes.io/projected/39bfb8a2-4ab0-478c-aa24-376a86fbdec4-kube-api-access-dkq8j\") pod \"ai-gateway-controller-5d9cd99f45-w9zmn\" (UID: \"39bfb8a2-4ab0-478c-aa24-376a86fbdec4\") " pod="opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn"
Oct 30 19:18:05.883559 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:05.883426    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39bfb8a2-4ab0-478c-aa24-376a86fbdec4-metrics-certs\") pod \"ai-gateway-controller-5d9cd99f45-w9zmn\" (UID: \"39bfb8a2-4ab0-478c-aa24-376a86fbdec4\") " pod="opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn"
Oct 30 19:18:05.885730 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:05.885705    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39bfb8a2-4ab0-478c-aa24-376a86fbdec4-metrics-certs\") pod \"ai-gateway-controller-5d9cd99f45-w9zmn\" (UID: \"39bfb8a2-4ab0-478c-aa24-376a86fbdec4\") " pod="opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn"
Oct 30 19:18:05.896490 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:05.896439    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkq8j\" (UniqueName: \"kubernetes.io/projected/39bfb8a2-4ab0-478c-aa24-376a86fbdec4-kube-api-access-dkq8j\") pod \"ai-gateway-controller-5d9cd99f45-w9zmn\" (UID: \"39bfb8a2-4ab0-478c-aa24-376a86fbdec4\") " pod="opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn"
Oct 30 19:18:05.966413 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:05.966392    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn"
Oct 30 19:18:06.245880 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:06.245857    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn"]
Oct 30 19:18:06.248589 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:18:06.248559    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39bfb8a2_4ab0_478c_aa24_376a86fbdec4.slice/crio-f9898cc9ab9547c959f50ab6b237db75223c4dd7b6a0ea504c521fca07ea20dc WatchSource:0}: Error finding container f9898cc9ab9547c959f50ab6b237db75223c4dd7b6a0ea504c521fca07ea20dc: Status 404 returned error can't find the container with id f9898cc9ab9547c959f50ab6b237db75223c4dd7b6a0ea504c521fca07ea20dc
Oct 30 19:18:06.581281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:06.581242    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn" event={"ID":"39bfb8a2-4ab0-478c-aa24-376a86fbdec4","Type":"ContainerStarted","Data":"f9898cc9ab9547c959f50ab6b237db75223c4dd7b6a0ea504c521fca07ea20dc"}
Oct 30 19:18:07.431756 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.431708    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"]
Oct 30 19:18:07.436287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.436258    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.440409 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.440369    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-vdgcw\""
Oct 30 19:18:07.458364 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.458320    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"]
Oct 30 19:18:07.458561 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.458386    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"]
Oct 30 19:18:07.463197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.462980    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.471995 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.471966    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"]
Oct 30 19:18:07.600010 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.599971    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.600199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.600020    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64b8f203-6e98-4ce3-adea-8686640b74e8-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.600199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.600056    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64b8f203-6e98-4ce3-adea-8686640b74e8-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.600199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.600088    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64b8f203-6e98-4ce3-adea-8686640b74e8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.600199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.600112    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.600199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.600141    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.600199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.600195    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.600468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.600226    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64b8f203-6e98-4ce3-adea-8686640b74e8-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.600468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.600254    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.600468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.600317    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64b8f203-6e98-4ce3-adea-8686640b74e8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.600468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.600344    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64b8f203-6e98-4ce3-adea-8686640b74e8-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.600468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.600385    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2zm2\" (UniqueName: \"kubernetes.io/projected/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-kube-api-access-v2zm2\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.600468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.600421    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64b8f203-6e98-4ce3-adea-8686640b74e8-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.600468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.600444    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.600736 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.600479    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64b8f203-6e98-4ce3-adea-8686640b74e8-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.600736 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.600502    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klvc5\" (UniqueName: \"kubernetes.io/projected/64b8f203-6e98-4ce3-adea-8686640b74e8-kube-api-access-klvc5\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.600736 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.600559    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.600736 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.600584    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.701987 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.701901    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64b8f203-6e98-4ce3-adea-8686640b74e8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.701987 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.701950    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.701987 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.701979    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.702279 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.702005    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.702279 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.702027    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64b8f203-6e98-4ce3-adea-8686640b74e8-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.702279 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.702054    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.702279 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.702113    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64b8f203-6e98-4ce3-adea-8686640b74e8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.702279 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.702146    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64b8f203-6e98-4ce3-adea-8686640b74e8-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.702518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.702288    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2zm2\" (UniqueName: \"kubernetes.io/projected/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-kube-api-access-v2zm2\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.702518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.702326    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64b8f203-6e98-4ce3-adea-8686640b74e8-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.702518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.702344    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.702518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.702369    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64b8f203-6e98-4ce3-adea-8686640b74e8-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.702518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.702396    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klvc5\" (UniqueName: \"kubernetes.io/projected/64b8f203-6e98-4ce3-adea-8686640b74e8-kube-api-access-klvc5\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.702518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.702444    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.702518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.702461    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.702518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.702487    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.702518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.702511    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64b8f203-6e98-4ce3-adea-8686640b74e8-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.702908 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.702556    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64b8f203-6e98-4ce3-adea-8686640b74e8-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.702908 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.702564    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64b8f203-6e98-4ce3-adea-8686640b74e8-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.702908 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.702742    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.702908 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.702788    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.703102 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.702992    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64b8f203-6e98-4ce3-adea-8686640b74e8-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.703157 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.703136    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.703807 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.703291    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64b8f203-6e98-4ce3-adea-8686640b74e8-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.703807 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.703525    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64b8f203-6e98-4ce3-adea-8686640b74e8-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.703807 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.703574    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64b8f203-6e98-4ce3-adea-8686640b74e8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.703807 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.703653    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.704053 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.704025    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.705794 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.705772    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64b8f203-6e98-4ce3-adea-8686640b74e8-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.706042 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.706025    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.706324 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.706303    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.706379 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.706309    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64b8f203-6e98-4ce3-adea-8686640b74e8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.714935 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.714891    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.715495 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.715451    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2zm2\" (UniqueName: \"kubernetes.io/projected/cb7c8396-fff1-47c2-9def-f721fb1eaeb4-kube-api-access-v2zm2\") pod \"maas-default-gateway-openshift-default-855f9574cf-bt7bg\" (UID: \"cb7c8396-fff1-47c2-9def-f721fb1eaeb4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:07.715647 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.715628    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klvc5\" (UniqueName: \"kubernetes.io/projected/64b8f203-6e98-4ce3-adea-8686640b74e8-kube-api-access-klvc5\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.716684 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.716661    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64b8f203-6e98-4ce3-adea-8686640b74e8-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-rc4rt\" (UID: \"64b8f203-6e98-4ce3-adea-8686640b74e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.761693 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.761666    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:07.784599 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:07.784571    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:08.059801 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:08.059771    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"]
Oct 30 19:18:08.061461 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:18:08.061430    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64b8f203_6e98_4ce3_adea_8686640b74e8.slice/crio-75d8850819f959ee50764affad228890618f227c4c79368e9ce22543675a9078 WatchSource:0}: Error finding container 75d8850819f959ee50764affad228890618f227c4c79368e9ce22543675a9078: Status 404 returned error can't find the container with id 75d8850819f959ee50764affad228890618f227c4c79368e9ce22543675a9078
Oct 30 19:18:08.088707 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:08.088681    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"]
Oct 30 19:18:08.094742 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:18:08.094699    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb7c8396_fff1_47c2_9def_f721fb1eaeb4.slice/crio-54909b8cb31c0ff1535b16cb3c805133003146b45ea9d444de1b7ad89490c382 WatchSource:0}: Error finding container 54909b8cb31c0ff1535b16cb3c805133003146b45ea9d444de1b7ad89490c382: Status 404 returned error can't find the container with id 54909b8cb31c0ff1535b16cb3c805133003146b45ea9d444de1b7ad89490c382
Oct 30 19:18:08.594686 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:08.594649    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg" event={"ID":"cb7c8396-fff1-47c2-9def-f721fb1eaeb4","Type":"ContainerStarted","Data":"54909b8cb31c0ff1535b16cb3c805133003146b45ea9d444de1b7ad89490c382"}
Oct 30 19:18:08.596657 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:08.596601    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt" event={"ID":"64b8f203-6e98-4ce3-adea-8686640b74e8","Type":"ContainerStarted","Data":"75d8850819f959ee50764affad228890618f227c4c79368e9ce22543675a9078"}
Oct 30 19:18:10.584458 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:10.584427    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Oct 30 19:18:11.625211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:11.622884    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn" event={"ID":"39bfb8a2-4ab0-478c-aa24-376a86fbdec4","Type":"ContainerStarted","Data":"738a62258e5a56135cf2c28aca24d72f48124a933032a6e1bc390565af27ade3"}
Oct 30 19:18:11.625211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:11.623158    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn"
Oct 30 19:18:11.635683 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:11.635647    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:18:11.635814 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:11.635728    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:18:11.635814 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:11.635773    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:18:11.646202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:11.646137    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn" podStartSLOduration=2.32586014 podStartE2EDuration="6.646125236s" totalImagesPullingTime="4.320265096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:11.644624612 +0000 UTC m=+723.385615690" watchObservedRunningTime="2026-10-30 19:18:11.646125236 +0000 UTC m=+723.387116315"
Oct 30 19:18:11.895379 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:11.895315    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:18:11.895490 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:11.895410    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:18:11.895490 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:11.895465    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:18:12.630038 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:12.629999    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg" event={"ID":"cb7c8396-fff1-47c2-9def-f721fb1eaeb4","Type":"ContainerStarted","Data":"4f754401ce6033b8481624256d9ebf5961a654bad6eeb9ac4e4d7b3270a627af"}
Oct 30 19:18:12.631854 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:12.631825    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt" event={"ID":"64b8f203-6e98-4ce3-adea-8686640b74e8","Type":"ContainerStarted","Data":"2040bb00f7b8bc426ccad90018bad7f545505d618125c5bbf8740906b5a75496"}
Oct 30 19:18:12.658885 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:12.658829    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg" podStartSLOduration=1.8617234759999999 podStartE2EDuration="5.658812504s" totalImagesPullingTime="3.797089028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:12.656309052 +0000 UTC m=+724.397300131" watchObservedRunningTime="2026-10-30 19:18:12.658812504 +0000 UTC m=+724.399803579"
Oct 30 19:18:12.692194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:12.691566    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt" podStartSLOduration=2.120727301 podStartE2EDuration="5.691544459s" totalImagesPullingTime="3.570817158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:12.689661717 +0000 UTC m=+724.430652797" watchObservedRunningTime="2026-10-30 19:18:12.691544459 +0000 UTC m=+724.432535539"
Oct 30 19:18:12.762018 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:12.761987    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:12.763246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:12.763219    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-rc4rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 19:18:12.763358 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:12.763260    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt" podUID="64b8f203-6e98-4ce3-adea-8686640b74e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 19:18:12.793194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:12.792732    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:12.797191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:12.796466    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-bt7bg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 19:18:12.797191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:12.796534    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg" podUID="cb7c8396-fff1-47c2-9def-f721fb1eaeb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 19:18:13.762342 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:13.762306    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-rc4rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 19:18:13.762757 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:13.762389    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt" podUID="64b8f203-6e98-4ce3-adea-8686640b74e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 19:18:13.790198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:13.787599    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-bt7bg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 19:18:13.790198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:13.787656    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg" podUID="cb7c8396-fff1-47c2-9def-f721fb1eaeb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 19:18:14.762687 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:14.762298    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-rc4rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 19:18:14.763211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:14.762716    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt" podUID="64b8f203-6e98-4ce3-adea-8686640b74e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 19:18:14.798202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:14.791292    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-bt7bg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 19:18:14.798202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:14.791378    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg" podUID="cb7c8396-fff1-47c2-9def-f721fb1eaeb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 19:18:15.636381 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:18:15.636346    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:18:15.763130 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:15.763094    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-rc4rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 19:18:15.763537 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:15.763156    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt" podUID="64b8f203-6e98-4ce3-adea-8686640b74e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 19:18:15.785887 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:15.785509    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-bt7bg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 19:18:15.785887 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:15.785565    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg" podUID="cb7c8396-fff1-47c2-9def-f721fb1eaeb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 19:18:16.763026 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:16.762994    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-rc4rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 19:18:16.763321 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:16.763283    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt" podUID="64b8f203-6e98-4ce3-adea-8686640b74e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 19:18:16.787214 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:16.786526    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-bt7bg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 19:18:16.787214 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:16.786583    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg" podUID="cb7c8396-fff1-47c2-9def-f721fb1eaeb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 19:18:17.762835 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:17.762799    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-rc4rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 19:18:17.763090 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:17.762851    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt" podUID="64b8f203-6e98-4ce3-adea-8686640b74e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 19:18:17.785515 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:17.785479    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-bt7bg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 19:18:17.786103 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:17.786074    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg" podUID="cb7c8396-fff1-47c2-9def-f721fb1eaeb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 19:18:18.762918 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:18.762881    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-rc4rt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Oct 30 19:18:18.763090 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:18.762949    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt" podUID="64b8f203-6e98-4ce3-adea-8686640b74e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Oct 30 19:18:18.785674 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:18.785645    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-bt7bg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body=
Oct 30 19:18:18.786016 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:18.785698    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg" podUID="cb7c8396-fff1-47c2-9def-f721fb1eaeb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused"
Oct 30 19:18:20.059088 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:20.059055    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:20.059683 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:20.059657    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:20.060495 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:20.060460    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-rc4rt"
Oct 30 19:18:20.275555 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:20.275466    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:20.276358 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:20.276077    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:20.276755 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:20.276713    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-bt7bg"
Oct 30 19:18:22.468797 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.468764    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"]
Oct 30 19:18:22.472703 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.472686    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.485792 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.485764    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr4pd\" (UniqueName: \"kubernetes.io/projected/39af7415-b31d-4bc0-8a67-ec591643f5c8-kube-api-access-xr4pd\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.485904 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.485829    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.485904 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.485889    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.486023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.485916    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.486023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.485948    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.486023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.485974    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.486199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.486023    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39af7415-b31d-4bc0-8a67-ec591643f5c8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.486199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.486061    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.486199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.486089    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.491836 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.491813    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"]
Oct 30 19:18:22.586835 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.586806    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.586982 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.586838    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.586982 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.586877    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr4pd\" (UniqueName: \"kubernetes.io/projected/39af7415-b31d-4bc0-8a67-ec591643f5c8-kube-api-access-xr4pd\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.586982 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.586920    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.586982 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.586949    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.586982 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.586973    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.587289 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.587005    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.587289 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.587029    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.587289 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.587069    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39af7415-b31d-4bc0-8a67-ec591643f5c8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.587451 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.587290    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.587451 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.587419    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.587451 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.587435    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.587594 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.587463    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.587902 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.587878    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39af7415-b31d-4bc0-8a67-ec591643f5c8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.589295 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.589275    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.589373 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.589335    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.598529 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.598502    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.598610 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.598543    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr4pd\" (UniqueName: \"kubernetes.io/projected/39af7415-b31d-4bc0-8a67-ec591643f5c8-kube-api-access-xr4pd\") pod \"maas-default-gateway-openshift-default-855f9574cf-lm7vd\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.633927 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.633906    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn"
Oct 30 19:18:22.785740 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.785663    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:22.911034 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.911000    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-6cf687fb7-fx8th"]
Oct 30 19:18:22.915713 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.915696    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-6cf687fb7-fx8th"
Oct 30 19:18:22.924710 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.924694    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-6cf687fb7-fx8th"]
Oct 30 19:18:22.991294 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.991264    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/669db161-f0cc-42e6-b9f6-8cff97eb4891-metrics-certs\") pod \"ai-gateway-controller-6cf687fb7-fx8th\" (UID: \"669db161-f0cc-42e6-b9f6-8cff97eb4891\") " pod="opendatahub/ai-gateway-controller-6cf687fb7-fx8th"
Oct 30 19:18:22.991435 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:22.991381    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv6z4\" (UniqueName: \"kubernetes.io/projected/669db161-f0cc-42e6-b9f6-8cff97eb4891-kube-api-access-mv6z4\") pod \"ai-gateway-controller-6cf687fb7-fx8th\" (UID: \"669db161-f0cc-42e6-b9f6-8cff97eb4891\") " pod="opendatahub/ai-gateway-controller-6cf687fb7-fx8th"
Oct 30 19:18:23.067903 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:23.067848    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"]
Oct 30 19:18:23.070991 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:18:23.070969    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39af7415_b31d_4bc0_8a67_ec591643f5c8.slice/crio-01b6df12aceeaaeab6063549c537626089388fb3646271774c0de32984452811 WatchSource:0}: Error finding container 01b6df12aceeaaeab6063549c537626089388fb3646271774c0de32984452811: Status 404 returned error can't find the container with id 01b6df12aceeaaeab6063549c537626089388fb3646271774c0de32984452811
Oct 30 19:18:23.074067 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:23.074033    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:18:23.074139 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:23.074094    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:18:23.074139 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:23.074121    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:18:23.091747 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:23.091720    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/669db161-f0cc-42e6-b9f6-8cff97eb4891-metrics-certs\") pod \"ai-gateway-controller-6cf687fb7-fx8th\" (UID: \"669db161-f0cc-42e6-b9f6-8cff97eb4891\") " pod="opendatahub/ai-gateway-controller-6cf687fb7-fx8th"
Oct 30 19:18:23.091828 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:23.091801    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv6z4\" (UniqueName: \"kubernetes.io/projected/669db161-f0cc-42e6-b9f6-8cff97eb4891-kube-api-access-mv6z4\") pod \"ai-gateway-controller-6cf687fb7-fx8th\" (UID: \"669db161-f0cc-42e6-b9f6-8cff97eb4891\") " pod="opendatahub/ai-gateway-controller-6cf687fb7-fx8th"
Oct 30 19:18:23.094041 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:23.094016    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/669db161-f0cc-42e6-b9f6-8cff97eb4891-metrics-certs\") pod \"ai-gateway-controller-6cf687fb7-fx8th\" (UID: \"669db161-f0cc-42e6-b9f6-8cff97eb4891\") " pod="opendatahub/ai-gateway-controller-6cf687fb7-fx8th"
Oct 30 19:18:23.103223 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:23.103204    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv6z4\" (UniqueName: \"kubernetes.io/projected/669db161-f0cc-42e6-b9f6-8cff97eb4891-kube-api-access-mv6z4\") pod \"ai-gateway-controller-6cf687fb7-fx8th\" (UID: \"669db161-f0cc-42e6-b9f6-8cff97eb4891\") " pod="opendatahub/ai-gateway-controller-6cf687fb7-fx8th"
Oct 30 19:18:23.230795 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:23.230765    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-6cf687fb7-fx8th"
Oct 30 19:18:23.519904 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:23.519882    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-6cf687fb7-fx8th"]
Oct 30 19:18:23.521876 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:18:23.521849    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod669db161_f0cc_42e6_b9f6_8cff97eb4891.slice/crio-427c37f4b95ab6b4630bc34a289b28c2b659412ef65aa64f5985cc9ae8e6cb5a WatchSource:0}: Error finding container 427c37f4b95ab6b4630bc34a289b28c2b659412ef65aa64f5985cc9ae8e6cb5a: Status 404 returned error can't find the container with id 427c37f4b95ab6b4630bc34a289b28c2b659412ef65aa64f5985cc9ae8e6cb5a
Oct 30 19:18:23.698111 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:23.698074    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd" event={"ID":"39af7415-b31d-4bc0-8a67-ec591643f5c8","Type":"ContainerStarted","Data":"36aac923525c3f0c5b28d22b373ecddb1ab45d7994b1114681459b78914b4dd8"}
Oct 30 19:18:23.698310 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:23.698140    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd" event={"ID":"39af7415-b31d-4bc0-8a67-ec591643f5c8","Type":"ContainerStarted","Data":"01b6df12aceeaaeab6063549c537626089388fb3646271774c0de32984452811"}
Oct 30 19:18:23.699333 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:23.699305    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-6cf687fb7-fx8th" event={"ID":"669db161-f0cc-42e6-b9f6-8cff97eb4891","Type":"ContainerStarted","Data":"427c37f4b95ab6b4630bc34a289b28c2b659412ef65aa64f5985cc9ae8e6cb5a"}
Oct 30 19:18:23.721458 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:23.721408    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd" podStartSLOduration=1.721393699 podStartE2EDuration="1.721393699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:23.719649049 +0000 UTC m=+735.460640129" watchObservedRunningTime="2026-10-30 19:18:23.721393699 +0000 UTC m=+735.462384782"
Oct 30 19:18:23.789426 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:23.789266    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:24.706525 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:24.706495    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-6cf687fb7-fx8th" event={"ID":"669db161-f0cc-42e6-b9f6-8cff97eb4891","Type":"ContainerStarted","Data":"6e566fb8298830d4747de27d03b10fd9256f2af96014228118f61c83c90cf2f0"}
Oct 30 19:18:24.706910 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:24.706571    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-6cf687fb7-fx8th"
Oct 30 19:18:24.725494 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:24.725446    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-6cf687fb7-fx8th" podStartSLOduration=2.289266358 podStartE2EDuration="2.725433102s" totalImagesPullingTime="436.166744ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:24.724133952 +0000 UTC m=+736.465125029" watchObservedRunningTime="2026-10-30 19:18:24.725433102 +0000 UTC m=+736.466424182"
Oct 30 19:18:24.789193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:24.788363    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-lm7vd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:18:24.789193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:24.788418    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd" podUID="39af7415-b31d-4bc0-8a67-ec591643f5c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 19:18:25.792195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:25.788605    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-lm7vd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:18:25.792195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:25.788660    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd" podUID="39af7415-b31d-4bc0-8a67-ec591643f5c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 19:18:26.788194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:26.787526    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-lm7vd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:18:26.788194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:26.787590    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd" podUID="39af7415-b31d-4bc0-8a67-ec591643f5c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 19:18:27.792350 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:27.792311    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-lm7vd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:18:27.792895 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:27.792375    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd" podUID="39af7415-b31d-4bc0-8a67-ec591643f5c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 19:18:28.791299 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:28.791257    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-lm7vd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:18:28.791594 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:28.791560    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd" podUID="39af7415-b31d-4bc0-8a67-ec591643f5c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 19:18:29.786751 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:29.786710    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-lm7vd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Oct 30 19:18:29.787144 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:29.786792    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd" podUID="39af7415-b31d-4bc0-8a67-ec591643f5c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Oct 30 19:18:31.313510 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:31.313477    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:31.314273 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:31.314247    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:31.315289 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:31.315270    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:18:35.714791 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:35.714762    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-6cf687fb7-fx8th"
Oct 30 19:18:35.762224 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:35.762191    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn"]
Oct 30 19:18:35.762426 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:35.762401    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn" podUID="39bfb8a2-4ab0-478c-aa24-376a86fbdec4" containerName="manager" containerID="cri-o://738a62258e5a56135cf2c28aca24d72f48124a933032a6e1bc390565af27ade3" gracePeriod=30
Oct 30 19:18:36.011822 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:36.011799    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn"
Oct 30 19:18:36.111189 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:36.111139    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/39bfb8a2-4ab0-478c-aa24-376a86fbdec4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/39bfb8a2-4ab0-478c-aa24-376a86fbdec4-metrics-certs\") pod \"39bfb8a2-4ab0-478c-aa24-376a86fbdec4\" (UID: \"39bfb8a2-4ab0-478c-aa24-376a86fbdec4\") "
Oct 30 19:18:36.111340 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:36.111317    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/39bfb8a2-4ab0-478c-aa24-376a86fbdec4-kube-api-access-dkq8j\" (UniqueName: \"kubernetes.io/projected/39bfb8a2-4ab0-478c-aa24-376a86fbdec4-kube-api-access-dkq8j\") pod \"39bfb8a2-4ab0-478c-aa24-376a86fbdec4\" (UID: \"39bfb8a2-4ab0-478c-aa24-376a86fbdec4\") "
Oct 30 19:18:36.113262 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:36.113225    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39bfb8a2-4ab0-478c-aa24-376a86fbdec4-metrics-certs" pod "39bfb8a2-4ab0-478c-aa24-376a86fbdec4" (UID: "39bfb8a2-4ab0-478c-aa24-376a86fbdec4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:18:36.113430 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:36.113408    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39bfb8a2-4ab0-478c-aa24-376a86fbdec4-kube-api-access-dkq8j" pod "39bfb8a2-4ab0-478c-aa24-376a86fbdec4" (UID: "39bfb8a2-4ab0-478c-aa24-376a86fbdec4"). InnerVolumeSpecName "kube-api-access-dkq8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:36.212180 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:36.212148    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkq8j\" (UniqueName: \"kubernetes.io/projected/39bfb8a2-4ab0-478c-aa24-376a86fbdec4-kube-api-access-dkq8j\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:36.212348 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:36.212201    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39bfb8a2-4ab0-478c-aa24-376a86fbdec4-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:36.773267 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:36.773163    4524 generic.go:356] "Generic (PLEG): container finished" podID="39bfb8a2-4ab0-478c-aa24-376a86fbdec4" containerID="738a62258e5a56135cf2c28aca24d72f48124a933032a6e1bc390565af27ade3" exitCode=0
Oct 30 19:18:36.773267 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:36.773205    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn" event={"ID":"39bfb8a2-4ab0-478c-aa24-376a86fbdec4","Type":"ContainerDied","Data":"738a62258e5a56135cf2c28aca24d72f48124a933032a6e1bc390565af27ade3"}
Oct 30 19:18:36.773267 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:36.773246    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn"
Oct 30 19:18:36.773267 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:36.773259    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn" event={"ID":"39bfb8a2-4ab0-478c-aa24-376a86fbdec4","Type":"ContainerDied","Data":"f9898cc9ab9547c959f50ab6b237db75223c4dd7b6a0ea504c521fca07ea20dc"}
Oct 30 19:18:36.773776 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:36.773281    4524 scope.go:122] "RemoveContainer" containerID="738a62258e5a56135cf2c28aca24d72f48124a933032a6e1bc390565af27ade3"
Oct 30 19:18:36.781883 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:36.781859    4524 scope.go:122] "RemoveContainer" containerID="738a62258e5a56135cf2c28aca24d72f48124a933032a6e1bc390565af27ade3"
Oct 30 19:18:36.782143 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:18:36.782123    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"738a62258e5a56135cf2c28aca24d72f48124a933032a6e1bc390565af27ade3\": container with ID starting with 738a62258e5a56135cf2c28aca24d72f48124a933032a6e1bc390565af27ade3 not found: ID does not exist" containerID="738a62258e5a56135cf2c28aca24d72f48124a933032a6e1bc390565af27ade3"
Oct 30 19:18:36.782220 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:36.782152    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"738a62258e5a56135cf2c28aca24d72f48124a933032a6e1bc390565af27ade3"} err="failed to get container status \"738a62258e5a56135cf2c28aca24d72f48124a933032a6e1bc390565af27ade3\": rpc error: code = NotFound desc = could not find container \"738a62258e5a56135cf2c28aca24d72f48124a933032a6e1bc390565af27ade3\": container with ID starting with 738a62258e5a56135cf2c28aca24d72f48124a933032a6e1bc390565af27ade3 not found: ID does not exist"
Oct 30 19:18:36.790441 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:36.790413    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn"]
Oct 30 19:18:36.794687 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:36.794668    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-5d9cd99f45-w9zmn"]
Oct 30 19:18:38.389649 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:18:38.389616    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="39bfb8a2-4ab0-478c-aa24-376a86fbdec4" path="/var/lib/kubelet/pods/39bfb8a2-4ab0-478c-aa24-376a86fbdec4/volumes"
Oct 30 19:18:45.624772 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:18:45.624732    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:19:09.418466 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.418436    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb"]
Oct 30 19:19:09.418933 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.418918    4524 cpu_manager.go:414] "containerMap: removing container" podUID="39bfb8a2-4ab0-478c-aa24-376a86fbdec4" containerName="manager"
Oct 30 19:19:09.418980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.418934    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="39bfb8a2-4ab0-478c-aa24-376a86fbdec4" containerName="manager"
Oct 30 19:19:09.419014 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.418997    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="39bfb8a2-4ab0-478c-aa24-376a86fbdec4" containerName="manager"
Oct 30 19:19:09.423226 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.423210    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb"
Oct 30 19:19:09.433319 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.433296    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-27b5d5-dockercfg-chfdc\""
Oct 30 19:19:09.434500 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.434479    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-27b5d5\""
Oct 30 19:19:09.450990 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.450961    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb"]
Oct 30 19:19:09.457070 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.457046    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h"]
Oct 30 19:19:09.460154 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.460137    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h"
Oct 30 19:19:09.479414 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.479387    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h"]
Oct 30 19:19:09.506283 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.506251    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db66fd6a-afe9-47e7-815c-5c66c86081d8-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h\" (UID: \"db66fd6a-afe9-47e7-815c-5c66c86081d8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h"
Oct 30 19:19:09.506443 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.506295    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-726kc\" (UniqueName: \"kubernetes.io/projected/1affe672-c981-43ad-bd2c-099648e4be4c-kube-api-access-726kc\") pod \"payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb\" (UID: \"1affe672-c981-43ad-bd2c-099648e4be4c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb"
Oct 30 19:19:09.506443 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.506431    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1affe672-c981-43ad-bd2c-099648e4be4c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb\" (UID: \"1affe672-c981-43ad-bd2c-099648e4be4c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb"
Oct 30 19:19:09.506556 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.506531    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjblp\" (UniqueName: \"kubernetes.io/projected/db66fd6a-afe9-47e7-815c-5c66c86081d8-kube-api-access-sjblp\") pod \"payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h\" (UID: \"db66fd6a-afe9-47e7-815c-5c66c86081d8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h"
Oct 30 19:19:09.608023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.607976    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjblp\" (UniqueName: \"kubernetes.io/projected/db66fd6a-afe9-47e7-815c-5c66c86081d8-kube-api-access-sjblp\") pod \"payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h\" (UID: \"db66fd6a-afe9-47e7-815c-5c66c86081d8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h"
Oct 30 19:19:09.608244 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.608102    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db66fd6a-afe9-47e7-815c-5c66c86081d8-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h\" (UID: \"db66fd6a-afe9-47e7-815c-5c66c86081d8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h"
Oct 30 19:19:09.608244 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.608129    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-726kc\" (UniqueName: \"kubernetes.io/projected/1affe672-c981-43ad-bd2c-099648e4be4c-kube-api-access-726kc\") pod \"payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb\" (UID: \"1affe672-c981-43ad-bd2c-099648e4be4c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb"
Oct 30 19:19:09.608244 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.608209    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1affe672-c981-43ad-bd2c-099648e4be4c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb\" (UID: \"1affe672-c981-43ad-bd2c-099648e4be4c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb"
Oct 30 19:19:09.609156 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.609124    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1affe672-c981-43ad-bd2c-099648e4be4c-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb\" (UID: \"1affe672-c981-43ad-bd2c-099648e4be4c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb"
Oct 30 19:19:09.609445 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.609422    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db66fd6a-afe9-47e7-815c-5c66c86081d8-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h\" (UID: \"db66fd6a-afe9-47e7-815c-5c66c86081d8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h"
Oct 30 19:19:09.620068 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.620044    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-726kc\" (UniqueName: \"kubernetes.io/projected/1affe672-c981-43ad-bd2c-099648e4be4c-kube-api-access-726kc\") pod \"payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb\" (UID: \"1affe672-c981-43ad-bd2c-099648e4be4c\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb"
Oct 30 19:19:09.620553 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.620530    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjblp\" (UniqueName: \"kubernetes.io/projected/db66fd6a-afe9-47e7-815c-5c66c86081d8-kube-api-access-sjblp\") pod \"payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h\" (UID: \"db66fd6a-afe9-47e7-815c-5c66c86081d8\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h"
Oct 30 19:19:09.740077 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.739989    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb"
Oct 30 19:19:09.782527 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.782493    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h"
Oct 30 19:19:09.957036 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.957008    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"]
Oct 30 19:19:09.961725 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.961705    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"
Oct 30 19:19:09.964663 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.964644    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-27b5d5-serving-cert\""
Oct 30 19:19:09.974088 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:09.974062    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"]
Oct 30 19:19:10.011779 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.011705    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7vns\" (UniqueName: \"kubernetes.io/projected/a03edaaa-eafc-4c25-846a-ff9fab080afc-kube-api-access-r7vns\") pod \"maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2\" (UID: \"a03edaaa-eafc-4c25-846a-ff9fab080afc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"
Oct 30 19:19:10.011902 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.011825    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a03edaaa-eafc-4c25-846a-ff9fab080afc-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2\" (UID: \"a03edaaa-eafc-4c25-846a-ff9fab080afc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"
Oct 30 19:19:10.011955 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.011919    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a03edaaa-eafc-4c25-846a-ff9fab080afc-maas-api-tls\") pod \"maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2\" (UID: \"a03edaaa-eafc-4c25-846a-ff9fab080afc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"
Oct 30 19:19:10.011998 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.011951    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a03edaaa-eafc-4c25-846a-ff9fab080afc-metrics-certs\") pod \"maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2\" (UID: \"a03edaaa-eafc-4c25-846a-ff9fab080afc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"
Oct 30 19:19:10.023096 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.023065    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb"]
Oct 30 19:19:10.025661 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:19:10.025625    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1affe672_c981_43ad_bd2c_099648e4be4c.slice/crio-cd1238d2336274f0c9a9e3896b64cfacd905f3e8928720a6f190b62b2e3f1e01 WatchSource:0}: Error finding container cd1238d2336274f0c9a9e3896b64cfacd905f3e8928720a6f190b62b2e3f1e01: Status 404 returned error can't find the container with id cd1238d2336274f0c9a9e3896b64cfacd905f3e8928720a6f190b62b2e3f1e01
Oct 30 19:19:10.079372 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.079347    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h"]
Oct 30 19:19:10.080309 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:19:10.080282    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb66fd6a_afe9_47e7_815c_5c66c86081d8.slice/crio-e106724ce31b84d9c11f7cae7645b5c906658ec8b14a66f6a3735480684da9a0 WatchSource:0}: Error finding container e106724ce31b84d9c11f7cae7645b5c906658ec8b14a66f6a3735480684da9a0: Status 404 returned error can't find the container with id e106724ce31b84d9c11f7cae7645b5c906658ec8b14a66f6a3735480684da9a0
Oct 30 19:19:10.112715 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.112694    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a03edaaa-eafc-4c25-846a-ff9fab080afc-maas-api-tls\") pod \"maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2\" (UID: \"a03edaaa-eafc-4c25-846a-ff9fab080afc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"
Oct 30 19:19:10.112830 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.112748    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a03edaaa-eafc-4c25-846a-ff9fab080afc-metrics-certs\") pod \"maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2\" (UID: \"a03edaaa-eafc-4c25-846a-ff9fab080afc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"
Oct 30 19:19:10.112879 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:19:10.112829    4524 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-serving-cert: secret "maas-api-e2e-worker-w4-27b5d5-serving-cert" not found
Oct 30 19:19:10.112879 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:19:10.112877    4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a03edaaa-eafc-4c25-846a-ff9fab080afc-maas-api-tls podName:a03edaaa-eafc-4c25-846a-ff9fab080afc nodeName:}" failed. No retries permitted until 2026-10-30 19:19:10.612862787 +0000 UTC m=+782.353853844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a03edaaa-eafc-4c25-846a-ff9fab080afc-maas-api-tls") pod "maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2" (UID: "a03edaaa-eafc-4c25-846a-ff9fab080afc") : secret "maas-api-e2e-worker-w4-27b5d5-serving-cert" not found
Oct 30 19:19:10.112996 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.112903    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7vns\" (UniqueName: \"kubernetes.io/projected/a03edaaa-eafc-4c25-846a-ff9fab080afc-kube-api-access-r7vns\") pod \"maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2\" (UID: \"a03edaaa-eafc-4c25-846a-ff9fab080afc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"
Oct 30 19:19:10.112996 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.112961    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a03edaaa-eafc-4c25-846a-ff9fab080afc-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2\" (UID: \"a03edaaa-eafc-4c25-846a-ff9fab080afc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"
Oct 30 19:19:10.113563 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.113542    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a03edaaa-eafc-4c25-846a-ff9fab080afc-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2\" (UID: \"a03edaaa-eafc-4c25-846a-ff9fab080afc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"
Oct 30 19:19:10.115361 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.115336    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a03edaaa-eafc-4c25-846a-ff9fab080afc-metrics-certs\") pod \"maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2\" (UID: \"a03edaaa-eafc-4c25-846a-ff9fab080afc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"
Oct 30 19:19:10.126147 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.126117    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7vns\" (UniqueName: \"kubernetes.io/projected/a03edaaa-eafc-4c25-846a-ff9fab080afc-kube-api-access-r7vns\") pod \"maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2\" (UID: \"a03edaaa-eafc-4c25-846a-ff9fab080afc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"
Oct 30 19:19:10.233233 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.233200    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd"]
Oct 30 19:19:10.238015 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.237955    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd"
Oct 30 19:19:10.244001 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.243973    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8"]
Oct 30 19:19:10.245392 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.244692    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-17e904\""
Oct 30 19:19:10.248287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.248267    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8"
Oct 30 19:19:10.256596 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.256225    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-17e904-dockercfg-6qst6\""
Oct 30 19:19:10.265756 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.265694    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8"]
Oct 30 19:19:10.266159 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.266137    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd"]
Oct 30 19:19:10.315145 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.315115    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79ldv\" (UniqueName: \"kubernetes.io/projected/fb610784-5f32-446f-9096-f524917ae3c7-kube-api-access-79ldv\") pod \"payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd\" (UID: \"fb610784-5f32-446f-9096-f524917ae3c7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd"
Oct 30 19:19:10.315283 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.315253    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c861815-5c5d-4a6a-9916-fcedf26bc4e0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8\" (UID: \"8c861815-5c5d-4a6a-9916-fcedf26bc4e0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8"
Oct 30 19:19:10.315344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.315328    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb610784-5f32-446f-9096-f524917ae3c7-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd\" (UID: \"fb610784-5f32-446f-9096-f524917ae3c7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd"
Oct 30 19:19:10.315467 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.315449    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ln5m\" (UniqueName: \"kubernetes.io/projected/8c861815-5c5d-4a6a-9916-fcedf26bc4e0-kube-api-access-5ln5m\") pod \"payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8\" (UID: \"8c861815-5c5d-4a6a-9916-fcedf26bc4e0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8"
Oct 30 19:19:10.416278 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.416245    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79ldv\" (UniqueName: \"kubernetes.io/projected/fb610784-5f32-446f-9096-f524917ae3c7-kube-api-access-79ldv\") pod \"payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd\" (UID: \"fb610784-5f32-446f-9096-f524917ae3c7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd"
Oct 30 19:19:10.416481 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.416326    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c861815-5c5d-4a6a-9916-fcedf26bc4e0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8\" (UID: \"8c861815-5c5d-4a6a-9916-fcedf26bc4e0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8"
Oct 30 19:19:10.416481 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.416389    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb610784-5f32-446f-9096-f524917ae3c7-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd\" (UID: \"fb610784-5f32-446f-9096-f524917ae3c7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd"
Oct 30 19:19:10.416481 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.416451    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ln5m\" (UniqueName: \"kubernetes.io/projected/8c861815-5c5d-4a6a-9916-fcedf26bc4e0-kube-api-access-5ln5m\") pod \"payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8\" (UID: \"8c861815-5c5d-4a6a-9916-fcedf26bc4e0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8"
Oct 30 19:19:10.417182 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.417146    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb610784-5f32-446f-9096-f524917ae3c7-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd\" (UID: \"fb610784-5f32-446f-9096-f524917ae3c7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd"
Oct 30 19:19:10.417312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.417210    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c861815-5c5d-4a6a-9916-fcedf26bc4e0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8\" (UID: \"8c861815-5c5d-4a6a-9916-fcedf26bc4e0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8"
Oct 30 19:19:10.428468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.428443    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79ldv\" (UniqueName: \"kubernetes.io/projected/fb610784-5f32-446f-9096-f524917ae3c7-kube-api-access-79ldv\") pod \"payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd\" (UID: \"fb610784-5f32-446f-9096-f524917ae3c7\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd"
Oct 30 19:19:10.428773 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.428489    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ln5m\" (UniqueName: \"kubernetes.io/projected/8c861815-5c5d-4a6a-9916-fcedf26bc4e0-kube-api-access-5ln5m\") pod \"payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8\" (UID: \"8c861815-5c5d-4a6a-9916-fcedf26bc4e0\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8"
Oct 30 19:19:10.552090 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.551775    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd"
Oct 30 19:19:10.562421 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.562398    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8"
Oct 30 19:19:10.644064 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.644016    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a03edaaa-eafc-4c25-846a-ff9fab080afc-maas-api-tls\") pod \"maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2\" (UID: \"a03edaaa-eafc-4c25-846a-ff9fab080afc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"
Oct 30 19:19:10.650324 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.650257    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a03edaaa-eafc-4c25-846a-ff9fab080afc-maas-api-tls\") pod \"maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2\" (UID: \"a03edaaa-eafc-4c25-846a-ff9fab080afc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"
Oct 30 19:19:10.883203 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.880068    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"
Oct 30 19:19:10.938483 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.938452    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd"]
Oct 30 19:19:10.941763 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.941250    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8"]
Oct 30 19:19:10.961516 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.961482    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h" event={"ID":"db66fd6a-afe9-47e7-815c-5c66c86081d8","Type":"ContainerStarted","Data":"e106724ce31b84d9c11f7cae7645b5c906658ec8b14a66f6a3735480684da9a0"}
Oct 30 19:19:10.964752 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.963384    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8" event={"ID":"8c861815-5c5d-4a6a-9916-fcedf26bc4e0","Type":"ContainerStarted","Data":"96a4ebc5ddb716811883f274d2bcb492a61658efc908917caaf9541b5f5b788f"}
Oct 30 19:19:10.981972 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.972133    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb" event={"ID":"1affe672-c981-43ad-bd2c-099648e4be4c","Type":"ContainerStarted","Data":"cd1238d2336274f0c9a9e3896b64cfacd905f3e8928720a6f190b62b2e3f1e01"}
Oct 30 19:19:10.981972 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:10.978935    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd" event={"ID":"fb610784-5f32-446f-9096-f524917ae3c7","Type":"ContainerStarted","Data":"4c877d6d1a2428beef92c87644835e2eb86eba59ecaa872fffff684dccabbcea"}
Oct 30 19:19:11.145625 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.145552    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5"]
Oct 30 19:19:11.150554 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.150533    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5"
Oct 30 19:19:11.166399 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.165351    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-cfa4bb-dockercfg-v6p7l\""
Oct 30 19:19:11.166399 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.165361    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-cfa4bb\""
Oct 30 19:19:11.167610 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.167521    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5"]
Oct 30 19:19:11.173228 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.173202    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d"]
Oct 30 19:19:11.177842 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.177821    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d"
Oct 30 19:19:11.191198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.191149    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d"]
Oct 30 19:19:11.210815 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.210766    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"]
Oct 30 19:19:11.215552 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:19:11.214830    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda03edaaa_eafc_4c25_846a_ff9fab080afc.slice/crio-1a30c391c2d46f57a4e2b48f8eabc22634d3dd4fe97c57be284e7a60b9f4615c WatchSource:0}: Error finding container 1a30c391c2d46f57a4e2b48f8eabc22634d3dd4fe97c57be284e7a60b9f4615c: Status 404 returned error can't find the container with id 1a30c391c2d46f57a4e2b48f8eabc22634d3dd4fe97c57be284e7a60b9f4615c
Oct 30 19:19:11.251959 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.251769    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34a1d722-a8a4-45e0-a5e4-6306c70eb7bb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d\" (UID: \"34a1d722-a8a4-45e0-a5e4-6306c70eb7bb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d"
Oct 30 19:19:11.251959 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.251848    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n65x9\" (UniqueName: \"kubernetes.io/projected/34a1d722-a8a4-45e0-a5e4-6306c70eb7bb-kube-api-access-n65x9\") pod \"payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d\" (UID: \"34a1d722-a8a4-45e0-a5e4-6306c70eb7bb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d"
Oct 30 19:19:11.251959 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.251870    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j958\" (UniqueName: \"kubernetes.io/projected/5355e2dd-ba58-4602-bedc-44a2979762e1-kube-api-access-7j958\") pod \"payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5\" (UID: \"5355e2dd-ba58-4602-bedc-44a2979762e1\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5"
Oct 30 19:19:11.251959 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.251892    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5355e2dd-ba58-4602-bedc-44a2979762e1-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5\" (UID: \"5355e2dd-ba58-4602-bedc-44a2979762e1\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5"
Oct 30 19:19:11.365499 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.365462    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34a1d722-a8a4-45e0-a5e4-6306c70eb7bb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d\" (UID: \"34a1d722-a8a4-45e0-a5e4-6306c70eb7bb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d"
Oct 30 19:19:11.365682 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.365540    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n65x9\" (UniqueName: \"kubernetes.io/projected/34a1d722-a8a4-45e0-a5e4-6306c70eb7bb-kube-api-access-n65x9\") pod \"payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d\" (UID: \"34a1d722-a8a4-45e0-a5e4-6306c70eb7bb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d"
Oct 30 19:19:11.365682 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.365566    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j958\" (UniqueName: \"kubernetes.io/projected/5355e2dd-ba58-4602-bedc-44a2979762e1-kube-api-access-7j958\") pod \"payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5\" (UID: \"5355e2dd-ba58-4602-bedc-44a2979762e1\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5"
Oct 30 19:19:11.365682 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.365584    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5355e2dd-ba58-4602-bedc-44a2979762e1-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5\" (UID: \"5355e2dd-ba58-4602-bedc-44a2979762e1\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5"
Oct 30 19:19:11.366441 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.366414    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34a1d722-a8a4-45e0-a5e4-6306c70eb7bb-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d\" (UID: \"34a1d722-a8a4-45e0-a5e4-6306c70eb7bb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d"
Oct 30 19:19:11.366510 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.366443    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5355e2dd-ba58-4602-bedc-44a2979762e1-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5\" (UID: \"5355e2dd-ba58-4602-bedc-44a2979762e1\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5"
Oct 30 19:19:11.378499 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.378469    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j958\" (UniqueName: \"kubernetes.io/projected/5355e2dd-ba58-4602-bedc-44a2979762e1-kube-api-access-7j958\") pod \"payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5\" (UID: \"5355e2dd-ba58-4602-bedc-44a2979762e1\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5"
Oct 30 19:19:11.378499 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.378475    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n65x9\" (UniqueName: \"kubernetes.io/projected/34a1d722-a8a4-45e0-a5e4-6306c70eb7bb-kube-api-access-n65x9\") pod \"payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d\" (UID: \"34a1d722-a8a4-45e0-a5e4-6306c70eb7bb\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d"
Oct 30 19:19:11.477487 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.477413    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5"
Oct 30 19:19:11.510209 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.510151    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d"
Oct 30 19:19:11.534326 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.534290    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq"]
Oct 30 19:19:11.539916 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.539898    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq"
Oct 30 19:19:11.544888 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.544393    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-2d5d53\""
Oct 30 19:19:11.566019 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.558031    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq"]
Oct 30 19:19:11.566019 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.562779    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh"]
Oct 30 19:19:11.566672 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.566650    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh"
Oct 30 19:19:11.581223 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.575358    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-2d5d53-dockercfg-d7wzl\""
Oct 30 19:19:11.603409 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.603380    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh"]
Oct 30 19:19:11.669350 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.668877    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ac2f9eb-a945-483a-9c0a-7ad91b8fd618-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq\" (UID: \"7ac2f9eb-a945-483a-9c0a-7ad91b8fd618\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq"
Oct 30 19:19:11.669350 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.669077    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3689eb31-fbba-4cbf-a355-7056ccec100b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh\" (UID: \"3689eb31-fbba-4cbf-a355-7056ccec100b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh"
Oct 30 19:19:11.669350 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.669218    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bw7d\" (UniqueName: \"kubernetes.io/projected/3689eb31-fbba-4cbf-a355-7056ccec100b-kube-api-access-9bw7d\") pod \"payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh\" (UID: \"3689eb31-fbba-4cbf-a355-7056ccec100b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh"
Oct 30 19:19:11.669604 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.669385    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmhfj\" (UniqueName: \"kubernetes.io/projected/7ac2f9eb-a945-483a-9c0a-7ad91b8fd618-kube-api-access-rmhfj\") pod \"payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq\" (UID: \"7ac2f9eb-a945-483a-9c0a-7ad91b8fd618\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq"
Oct 30 19:19:11.785045 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.780187    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ac2f9eb-a945-483a-9c0a-7ad91b8fd618-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq\" (UID: \"7ac2f9eb-a945-483a-9c0a-7ad91b8fd618\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq"
Oct 30 19:19:11.785045 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.780259    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3689eb31-fbba-4cbf-a355-7056ccec100b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh\" (UID: \"3689eb31-fbba-4cbf-a355-7056ccec100b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh"
Oct 30 19:19:11.785045 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.780296    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bw7d\" (UniqueName: \"kubernetes.io/projected/3689eb31-fbba-4cbf-a355-7056ccec100b-kube-api-access-9bw7d\") pod \"payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh\" (UID: \"3689eb31-fbba-4cbf-a355-7056ccec100b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh"
Oct 30 19:19:11.785045 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.780441    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmhfj\" (UniqueName: \"kubernetes.io/projected/7ac2f9eb-a945-483a-9c0a-7ad91b8fd618-kube-api-access-rmhfj\") pod \"payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq\" (UID: \"7ac2f9eb-a945-483a-9c0a-7ad91b8fd618\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq"
Oct 30 19:19:11.785045 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.781684    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ac2f9eb-a945-483a-9c0a-7ad91b8fd618-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq\" (UID: \"7ac2f9eb-a945-483a-9c0a-7ad91b8fd618\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq"
Oct 30 19:19:11.785045 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.782443    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3689eb31-fbba-4cbf-a355-7056ccec100b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh\" (UID: \"3689eb31-fbba-4cbf-a355-7056ccec100b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh"
Oct 30 19:19:11.810630 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.810567    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmhfj\" (UniqueName: \"kubernetes.io/projected/7ac2f9eb-a945-483a-9c0a-7ad91b8fd618-kube-api-access-rmhfj\") pod \"payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq\" (UID: \"7ac2f9eb-a945-483a-9c0a-7ad91b8fd618\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq"
Oct 30 19:19:11.828385 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.828280    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bw7d\" (UniqueName: \"kubernetes.io/projected/3689eb31-fbba-4cbf-a355-7056ccec100b-kube-api-access-9bw7d\") pod \"payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh\" (UID: \"3689eb31-fbba-4cbf-a355-7056ccec100b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh"
Oct 30 19:19:11.865313 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.865279    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5"]
Oct 30 19:19:11.878636 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.878443    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq"
Oct 30 19:19:11.908180 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.908130    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh"
Oct 30 19:19:11.911645 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:11.911600    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d"]
Oct 30 19:19:12.009437 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.007426    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c"]
Oct 30 19:19:12.027932 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.017573    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c"
Oct 30 19:19:12.027932 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.026974    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2" event={"ID":"a03edaaa-eafc-4c25-846a-ff9fab080afc","Type":"ContainerStarted","Data":"1a30c391c2d46f57a4e2b48f8eabc22634d3dd4fe97c57be284e7a60b9f4615c"}
Oct 30 19:19:12.035343 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.031952    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w6-bce71f-dockercfg-77zgh\""
Oct 30 19:19:12.035343 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.033796    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5" event={"ID":"5355e2dd-ba58-4602-bedc-44a2979762e1","Type":"ContainerStarted","Data":"9f1bedacef16f0852f3be9b14b41dac5933a987ca943e33bb7c7ce3ecc95b0e8"}
Oct 30 19:19:12.044625 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.044556    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d" event={"ID":"34a1d722-a8a4-45e0-a5e4-6306c70eb7bb","Type":"ContainerStarted","Data":"265f1b8d6db3a93255c112692d0cbf87b0474c151d991158a15294ba9ea9cb51"}
Oct 30 19:19:12.046668 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.046604    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w6-bce71f\""
Oct 30 19:19:12.051256 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.051106    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c"]
Oct 30 19:19:12.190194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.115246    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whq2r\" (UniqueName: \"kubernetes.io/projected/4bac4098-9dc9-4305-b0ef-286d82cae682-kube-api-access-whq2r\") pod \"payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c\" (UID: \"4bac4098-9dc9-4305-b0ef-286d82cae682\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c"
Oct 30 19:19:12.190194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.115311    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4bac4098-9dc9-4305-b0ef-286d82cae682-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c\" (UID: \"4bac4098-9dc9-4305-b0ef-286d82cae682\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c"
Oct 30 19:19:12.190194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.167410    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"]
Oct 30 19:19:12.190494 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.190334    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"
Oct 30 19:19:12.209134 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.207670    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"]
Oct 30 19:19:12.258189 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.233525    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-39b867-serving-cert\""
Oct 30 19:19:12.264799 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.264702    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whq2r\" (UniqueName: \"kubernetes.io/projected/4bac4098-9dc9-4305-b0ef-286d82cae682-kube-api-access-whq2r\") pod \"payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c\" (UID: \"4bac4098-9dc9-4305-b0ef-286d82cae682\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c"
Oct 30 19:19:12.265963 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.265929    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4bac4098-9dc9-4305-b0ef-286d82cae682-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c\" (UID: \"4bac4098-9dc9-4305-b0ef-286d82cae682\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c"
Oct 30 19:19:12.267159 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.267136    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4bac4098-9dc9-4305-b0ef-286d82cae682-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c\" (UID: \"4bac4098-9dc9-4305-b0ef-286d82cae682\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c"
Oct 30 19:19:12.280026 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:19:12.279304    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ac2f9eb_a945_483a_9c0a_7ad91b8fd618.slice/crio-675a04c8942050541aa10d69161a29aef1ae8d8ab722d796fa33c134059f18a3 WatchSource:0}: Error finding container 675a04c8942050541aa10d69161a29aef1ae8d8ab722d796fa33c134059f18a3: Status 404 returned error can't find the container with id 675a04c8942050541aa10d69161a29aef1ae8d8ab722d796fa33c134059f18a3
Oct 30 19:19:12.300287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.300260    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh"]
Oct 30 19:19:12.300448 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:19:12.300432    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3689eb31_fbba_4cbf_a355_7056ccec100b.slice/crio-a01728676bb46d5df816bd641a4878ec839ef0a00bd60b23ab05c943e5c8a2a9 WatchSource:0}: Error finding container a01728676bb46d5df816bd641a4878ec839ef0a00bd60b23ab05c943e5c8a2a9: Status 404 returned error can't find the container with id a01728676bb46d5df816bd641a4878ec839ef0a00bd60b23ab05c943e5c8a2a9
Oct 30 19:19:12.304610 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.304590    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq"]
Oct 30 19:19:12.313699 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.313674    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whq2r\" (UniqueName: \"kubernetes.io/projected/4bac4098-9dc9-4305-b0ef-286d82cae682-kube-api-access-whq2r\") pod \"payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c\" (UID: \"4bac4098-9dc9-4305-b0ef-286d82cae682\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c"
Oct 30 19:19:12.349934 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.349503    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c"
Oct 30 19:19:12.367943 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.367911    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk22c\" (UniqueName: \"kubernetes.io/projected/68c74fd5-f30e-4eb7-bc88-e2b679a28451-kube-api-access-hk22c\") pod \"maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9\" (UID: \"68c74fd5-f30e-4eb7-bc88-e2b679a28451\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"
Oct 30 19:19:12.368053 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.367984    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68c74fd5-f30e-4eb7-bc88-e2b679a28451-maas-api-tls\") pod \"maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9\" (UID: \"68c74fd5-f30e-4eb7-bc88-e2b679a28451\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"
Oct 30 19:19:12.368132 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.368081    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68c74fd5-f30e-4eb7-bc88-e2b679a28451-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9\" (UID: \"68c74fd5-f30e-4eb7-bc88-e2b679a28451\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"
Oct 30 19:19:12.368132 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.368105    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68c74fd5-f30e-4eb7-bc88-e2b679a28451-metrics-certs\") pod \"maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9\" (UID: \"68c74fd5-f30e-4eb7-bc88-e2b679a28451\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"
Oct 30 19:19:12.470502 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.469616    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68c74fd5-f30e-4eb7-bc88-e2b679a28451-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9\" (UID: \"68c74fd5-f30e-4eb7-bc88-e2b679a28451\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"
Oct 30 19:19:12.470502 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.469648    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68c74fd5-f30e-4eb7-bc88-e2b679a28451-metrics-certs\") pod \"maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9\" (UID: \"68c74fd5-f30e-4eb7-bc88-e2b679a28451\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"
Oct 30 19:19:12.470502 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.469690    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk22c\" (UniqueName: \"kubernetes.io/projected/68c74fd5-f30e-4eb7-bc88-e2b679a28451-kube-api-access-hk22c\") pod \"maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9\" (UID: \"68c74fd5-f30e-4eb7-bc88-e2b679a28451\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"
Oct 30 19:19:12.470502 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.469732    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68c74fd5-f30e-4eb7-bc88-e2b679a28451-maas-api-tls\") pod \"maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9\" (UID: \"68c74fd5-f30e-4eb7-bc88-e2b679a28451\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"
Oct 30 19:19:12.470502 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.470061    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68c74fd5-f30e-4eb7-bc88-e2b679a28451-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9\" (UID: \"68c74fd5-f30e-4eb7-bc88-e2b679a28451\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"
Oct 30 19:19:12.493990 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.493931    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk22c\" (UniqueName: \"kubernetes.io/projected/68c74fd5-f30e-4eb7-bc88-e2b679a28451-kube-api-access-hk22c\") pod \"maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9\" (UID: \"68c74fd5-f30e-4eb7-bc88-e2b679a28451\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"
Oct 30 19:19:12.497688 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.497662    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68c74fd5-f30e-4eb7-bc88-e2b679a28451-maas-api-tls\") pod \"maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9\" (UID: \"68c74fd5-f30e-4eb7-bc88-e2b679a28451\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"
Oct 30 19:19:12.500089 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.500044    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68c74fd5-f30e-4eb7-bc88-e2b679a28451-metrics-certs\") pod \"maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9\" (UID: \"68c74fd5-f30e-4eb7-bc88-e2b679a28451\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"
Oct 30 19:19:12.578612 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.578529    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"
Oct 30 19:19:12.704028 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.703997    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88"]
Oct 30 19:19:12.721535 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.721506    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c"]
Oct 30 19:19:12.723525 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.721701    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88"
Oct 30 19:19:12.730982 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.730611    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88"]
Oct 30 19:19:12.730982 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.730690    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-ad23c7\""
Oct 30 19:19:12.776744 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.776501    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0806e982-fe3c-496d-95e2-c75d2b23dacf-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88\" (UID: \"0806e982-fe3c-496d-95e2-c75d2b23dacf\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88"
Oct 30 19:19:12.776744 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.776638    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfn5d\" (UniqueName: \"kubernetes.io/projected/0806e982-fe3c-496d-95e2-c75d2b23dacf-kube-api-access-tfn5d\") pod \"payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88\" (UID: \"0806e982-fe3c-496d-95e2-c75d2b23dacf\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88"
Oct 30 19:19:12.880431 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.878790    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfn5d\" (UniqueName: \"kubernetes.io/projected/0806e982-fe3c-496d-95e2-c75d2b23dacf-kube-api-access-tfn5d\") pod \"payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88\" (UID: \"0806e982-fe3c-496d-95e2-c75d2b23dacf\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88"
Oct 30 19:19:12.880431 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.878878    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0806e982-fe3c-496d-95e2-c75d2b23dacf-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88\" (UID: \"0806e982-fe3c-496d-95e2-c75d2b23dacf\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88"
Oct 30 19:19:12.880431 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.879909    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0806e982-fe3c-496d-95e2-c75d2b23dacf-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88\" (UID: \"0806e982-fe3c-496d-95e2-c75d2b23dacf\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88"
Oct 30 19:19:12.984783 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.984733    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"]
Oct 30 19:19:13.010351 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:12.985907    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfn5d\" (UniqueName: \"kubernetes.io/projected/0806e982-fe3c-496d-95e2-c75d2b23dacf-kube-api-access-tfn5d\") pod \"payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88\" (UID: \"0806e982-fe3c-496d-95e2-c75d2b23dacf\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88"
Oct 30 19:19:13.010351 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:19:13.005998    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68c74fd5_f30e_4eb7_bc88_e2b679a28451.slice/crio-267b89d84b7b78fddb58d3a5813f60018b39a932223d58480b609aa66e3b3659 WatchSource:0}: Error finding container 267b89d84b7b78fddb58d3a5813f60018b39a932223d58480b609aa66e3b3659: Status 404 returned error can't find the container with id 267b89d84b7b78fddb58d3a5813f60018b39a932223d58480b609aa66e3b3659
Oct 30 19:19:13.018918 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.018894    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"]
Oct 30 19:19:13.019033 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.018957    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"]
Oct 30 19:19:13.019227 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.019206    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"
Oct 30 19:19:13.025337 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.025165    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-17e904-serving-cert\""
Oct 30 19:19:13.077345 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.077316    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88"
Oct 30 19:19:13.087228 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.086194    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/969098da-cc93-4414-b2e8-08f2f4ccf4e9-maas-api-tls\") pod \"maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq\" (UID: \"969098da-cc93-4414-b2e8-08f2f4ccf4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"
Oct 30 19:19:13.087228 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.086393    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/969098da-cc93-4414-b2e8-08f2f4ccf4e9-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq\" (UID: \"969098da-cc93-4414-b2e8-08f2f4ccf4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"
Oct 30 19:19:13.087228 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.086590    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz4v7\" (UniqueName: \"kubernetes.io/projected/969098da-cc93-4414-b2e8-08f2f4ccf4e9-kube-api-access-hz4v7\") pod \"maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq\" (UID: \"969098da-cc93-4414-b2e8-08f2f4ccf4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"
Oct 30 19:19:13.087228 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.086720    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/969098da-cc93-4414-b2e8-08f2f4ccf4e9-metrics-certs\") pod \"maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq\" (UID: \"969098da-cc93-4414-b2e8-08f2f4ccf4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"
Oct 30 19:19:13.091747 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.091011    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c" event={"ID":"4bac4098-9dc9-4305-b0ef-286d82cae682","Type":"ContainerStarted","Data":"3d482526290526ca442ae22519a0e387c9c282ff70a3e165c5705a379abede8f"}
Oct 30 19:19:13.155710 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.155538    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2" event={"ID":"a03edaaa-eafc-4c25-846a-ff9fab080afc","Type":"ContainerStarted","Data":"10fe9cce3096067886f9ef3cedda546a783e986b099571720a0a862d2e05e990"}
Oct 30 19:19:13.157242 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.157104    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"
Oct 30 19:19:13.185927 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.185806    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2" podStartSLOduration=3.502635461 podStartE2EDuration="4.185788242s" totalImagesPullingTime="683.152781ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:13.184442131 +0000 UTC m=+784.925433212" watchObservedRunningTime="2026-10-30 19:19:13.185788242 +0000 UTC m=+784.926779325"
Oct 30 19:19:13.187249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.187224    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9" event={"ID":"68c74fd5-f30e-4eb7-bc88-e2b679a28451","Type":"ContainerStarted","Data":"267b89d84b7b78fddb58d3a5813f60018b39a932223d58480b609aa66e3b3659"}
Oct 30 19:19:13.221088 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.221055    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq" event={"ID":"7ac2f9eb-a945-483a-9c0a-7ad91b8fd618","Type":"ContainerStarted","Data":"675a04c8942050541aa10d69161a29aef1ae8d8ab722d796fa33c134059f18a3"}
Oct 30 19:19:13.235032 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.234932    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh" event={"ID":"3689eb31-fbba-4cbf-a355-7056ccec100b","Type":"ContainerStarted","Data":"a01728676bb46d5df816bd641a4878ec839ef0a00bd60b23ab05c943e5c8a2a9"}
Oct 30 19:19:13.245920 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.245889    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/969098da-cc93-4414-b2e8-08f2f4ccf4e9-metrics-certs\") pod \"maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq\" (UID: \"969098da-cc93-4414-b2e8-08f2f4ccf4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"
Oct 30 19:19:13.251252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.251228    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/969098da-cc93-4414-b2e8-08f2f4ccf4e9-maas-api-tls\") pod \"maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq\" (UID: \"969098da-cc93-4414-b2e8-08f2f4ccf4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"
Oct 30 19:19:13.251442 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.251425    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/969098da-cc93-4414-b2e8-08f2f4ccf4e9-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq\" (UID: \"969098da-cc93-4414-b2e8-08f2f4ccf4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"
Oct 30 19:19:13.251704 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.251687    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz4v7\" (UniqueName: \"kubernetes.io/projected/969098da-cc93-4414-b2e8-08f2f4ccf4e9-kube-api-access-hz4v7\") pod \"maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq\" (UID: \"969098da-cc93-4414-b2e8-08f2f4ccf4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"
Oct 30 19:19:13.252673 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.252652    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/969098da-cc93-4414-b2e8-08f2f4ccf4e9-metrics-certs\") pod \"maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq\" (UID: \"969098da-cc93-4414-b2e8-08f2f4ccf4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"
Oct 30 19:19:13.252772 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.252239    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/969098da-cc93-4414-b2e8-08f2f4ccf4e9-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq\" (UID: \"969098da-cc93-4414-b2e8-08f2f4ccf4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"
Oct 30 19:19:13.258856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.258811    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/969098da-cc93-4414-b2e8-08f2f4ccf4e9-maas-api-tls\") pod \"maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq\" (UID: \"969098da-cc93-4414-b2e8-08f2f4ccf4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"
Oct 30 19:19:13.287318 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.287234    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz4v7\" (UniqueName: \"kubernetes.io/projected/969098da-cc93-4414-b2e8-08f2f4ccf4e9-kube-api-access-hz4v7\") pod \"maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq\" (UID: \"969098da-cc93-4414-b2e8-08f2f4ccf4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"
Oct 30 19:19:13.376386 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.376155    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"
Oct 30 19:19:13.522921 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.522892    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88"]
Oct 30 19:19:13.732280 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.732202    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"]
Oct 30 19:19:13.765195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.765153    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"]
Oct 30 19:19:13.771146 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.771127    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"
Oct 30 19:19:13.789297 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.788687    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dg4v\" (UniqueName: \"kubernetes.io/projected/fbb268b8-644a-41c8-a78b-2779914d0e9d-kube-api-access-4dg4v\") pod \"maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr\" (UID: \"fbb268b8-644a-41c8-a78b-2779914d0e9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"
Oct 30 19:19:13.789297 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.788749    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbb268b8-644a-41c8-a78b-2779914d0e9d-maas-api-tls\") pod \"maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr\" (UID: \"fbb268b8-644a-41c8-a78b-2779914d0e9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"
Oct 30 19:19:13.789297 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.788791    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb268b8-644a-41c8-a78b-2779914d0e9d-metrics-certs\") pod \"maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr\" (UID: \"fbb268b8-644a-41c8-a78b-2779914d0e9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"
Oct 30 19:19:13.789297 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.788844    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbb268b8-644a-41c8-a78b-2779914d0e9d-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr\" (UID: \"fbb268b8-644a-41c8-a78b-2779914d0e9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"
Oct 30 19:19:13.796413 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.796389    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"]
Oct 30 19:19:13.847216 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.847187    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-cfa4bb-serving-cert\""
Oct 30 19:19:13.927627 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.926245    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dg4v\" (UniqueName: \"kubernetes.io/projected/fbb268b8-644a-41c8-a78b-2779914d0e9d-kube-api-access-4dg4v\") pod \"maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr\" (UID: \"fbb268b8-644a-41c8-a78b-2779914d0e9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"
Oct 30 19:19:13.927627 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.926339    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbb268b8-644a-41c8-a78b-2779914d0e9d-maas-api-tls\") pod \"maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr\" (UID: \"fbb268b8-644a-41c8-a78b-2779914d0e9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"
Oct 30 19:19:13.927627 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.926402    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb268b8-644a-41c8-a78b-2779914d0e9d-metrics-certs\") pod \"maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr\" (UID: \"fbb268b8-644a-41c8-a78b-2779914d0e9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"
Oct 30 19:19:13.927627 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.926481    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbb268b8-644a-41c8-a78b-2779914d0e9d-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr\" (UID: \"fbb268b8-644a-41c8-a78b-2779914d0e9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"
Oct 30 19:19:13.927627 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.927267    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbb268b8-644a-41c8-a78b-2779914d0e9d-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr\" (UID: \"fbb268b8-644a-41c8-a78b-2779914d0e9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"
Oct 30 19:19:13.943040 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.942967    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbb268b8-644a-41c8-a78b-2779914d0e9d-maas-api-tls\") pod \"maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr\" (UID: \"fbb268b8-644a-41c8-a78b-2779914d0e9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"
Oct 30 19:19:13.944551 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.944521    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb268b8-644a-41c8-a78b-2779914d0e9d-metrics-certs\") pod \"maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr\" (UID: \"fbb268b8-644a-41c8-a78b-2779914d0e9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"
Oct 30 19:19:13.960435 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:13.960397    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dg4v\" (UniqueName: \"kubernetes.io/projected/fbb268b8-644a-41c8-a78b-2779914d0e9d-kube-api-access-4dg4v\") pod \"maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr\" (UID: \"fbb268b8-644a-41c8-a78b-2779914d0e9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"
Oct 30 19:19:14.125507 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:14.125203    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"
Oct 30 19:19:14.258008 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:14.257928    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88" event={"ID":"0806e982-fe3c-496d-95e2-c75d2b23dacf","Type":"ContainerStarted","Data":"384549fc4b4caf041bb8960f564edd12f9a5fd18eeebe7b172b41227dc2d60fe"}
Oct 30 19:19:14.261082 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:14.261032    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9" event={"ID":"68c74fd5-f30e-4eb7-bc88-e2b679a28451","Type":"ContainerStarted","Data":"eb88dac7d0077e079f9e895199efa5af9e6a4285a5dee8aa98b842c92b34791a"}
Oct 30 19:19:14.285954 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:14.284418    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9" podStartSLOduration=1.5931348600000002 podStartE2EDuration="2.284399344s" totalImagesPullingTime="691.264484ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:14.283586209 +0000 UTC m=+786.024577290" watchObservedRunningTime="2026-10-30 19:19:14.284399344 +0000 UTC m=+786.025390427"
Oct 30 19:19:14.288354 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:14.288270    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq" event={"ID":"969098da-cc93-4414-b2e8-08f2f4ccf4e9","Type":"ContainerStarted","Data":"de971166b50d0581bd1a14491ec32dc99e4928488a50ce177a2cc7a25b6507c4"}
Oct 30 19:19:14.541932 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:19:14.541906    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbb268b8_644a_41c8_a78b_2779914d0e9d.slice/crio-3cbd6ce2cf6aa2885222397eabea6eae4e30d6f8cf6e31ad76deca35d60a3c90 WatchSource:0}: Error finding container 3cbd6ce2cf6aa2885222397eabea6eae4e30d6f8cf6e31ad76deca35d60a3c90: Status 404 returned error can't find the container with id 3cbd6ce2cf6aa2885222397eabea6eae4e30d6f8cf6e31ad76deca35d60a3c90
Oct 30 19:19:14.550930 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:14.550908    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"]
Oct 30 19:19:15.332315 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.324398    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr" event={"ID":"fbb268b8-644a-41c8-a78b-2779914d0e9d","Type":"ContainerStarted","Data":"fdcf6fb1b03028e373db4d39d4a5f86ab67afd01659d86a44a9b934e65a6c131"}
Oct 30 19:19:15.332315 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.324463    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr" event={"ID":"fbb268b8-644a-41c8-a78b-2779914d0e9d","Type":"ContainerStarted","Data":"3cbd6ce2cf6aa2885222397eabea6eae4e30d6f8cf6e31ad76deca35d60a3c90"}
Oct 30 19:19:15.332315 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.324517    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"
Oct 30 19:19:15.332315 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.328778    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq" event={"ID":"969098da-cc93-4414-b2e8-08f2f4ccf4e9","Type":"ContainerStarted","Data":"43b7487604a6a0bb69ee51b6c3335febb06a3f44441c1d1d63117a7cc8e7a1aa"}
Oct 30 19:19:15.332315 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.328837    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"
Oct 30 19:19:15.332315 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.328865    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"
Oct 30 19:19:15.362774 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.359093    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr" podStartSLOduration=1.889399115 podStartE2EDuration="2.358904814s" totalImagesPullingTime="469.505699ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:15.348304392 +0000 UTC m=+787.089295503" watchObservedRunningTime="2026-10-30 19:19:15.358904814 +0000 UTC m=+787.099895894"
Oct 30 19:19:15.624008 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.623861    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq" podStartSLOduration=3.096740539 podStartE2EDuration="3.623840354s" totalImagesPullingTime="527.099815ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:15.38152556 +0000 UTC m=+787.122516634" watchObservedRunningTime="2026-10-30 19:19:15.623840354 +0000 UTC m=+787.364831437"
Oct 30 19:19:15.627033 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.627006    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"]
Oct 30 19:19:15.637414 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.637389    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"
Oct 30 19:19:15.654190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.645678    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-2d5d53-serving-cert\""
Oct 30 19:19:15.664028 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.663662    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29e3ef22-7657-4fe5-803e-75a93978aced-maas-api-tls\") pod \"maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g\" (UID: \"29e3ef22-7657-4fe5-803e-75a93978aced\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"
Oct 30 19:19:15.664028 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.663706    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29e3ef22-7657-4fe5-803e-75a93978aced-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g\" (UID: \"29e3ef22-7657-4fe5-803e-75a93978aced\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"
Oct 30 19:19:15.664028 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.663774    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29e3ef22-7657-4fe5-803e-75a93978aced-metrics-certs\") pod \"maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g\" (UID: \"29e3ef22-7657-4fe5-803e-75a93978aced\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"
Oct 30 19:19:15.664028 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.663806    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn2tp\" (UniqueName: \"kubernetes.io/projected/29e3ef22-7657-4fe5-803e-75a93978aced-kube-api-access-wn2tp\") pod \"maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g\" (UID: \"29e3ef22-7657-4fe5-803e-75a93978aced\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"
Oct 30 19:19:15.665335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.665314    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"]
Oct 30 19:19:15.701045 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:19:15.700981    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:19:15.767152 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.766283    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29e3ef22-7657-4fe5-803e-75a93978aced-maas-api-tls\") pod \"maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g\" (UID: \"29e3ef22-7657-4fe5-803e-75a93978aced\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"
Oct 30 19:19:15.767152 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.766333    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29e3ef22-7657-4fe5-803e-75a93978aced-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g\" (UID: \"29e3ef22-7657-4fe5-803e-75a93978aced\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"
Oct 30 19:19:15.767152 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:19:15.766382    4524 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-serving-cert: secret "maas-api-e2e-worker-w5-2d5d53-serving-cert" not found
Oct 30 19:19:15.767152 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:19:15.766483    4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29e3ef22-7657-4fe5-803e-75a93978aced-maas-api-tls podName:29e3ef22-7657-4fe5-803e-75a93978aced nodeName:}" failed. No retries permitted until 2026-10-30 19:19:16.266461912 +0000 UTC m=+788.007452983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/29e3ef22-7657-4fe5-803e-75a93978aced-maas-api-tls") pod "maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g" (UID: "29e3ef22-7657-4fe5-803e-75a93978aced") : secret "maas-api-e2e-worker-w5-2d5d53-serving-cert" not found
Oct 30 19:19:15.767152 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.766411    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29e3ef22-7657-4fe5-803e-75a93978aced-metrics-certs\") pod \"maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g\" (UID: \"29e3ef22-7657-4fe5-803e-75a93978aced\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"
Oct 30 19:19:15.767152 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.766815    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn2tp\" (UniqueName: \"kubernetes.io/projected/29e3ef22-7657-4fe5-803e-75a93978aced-kube-api-access-wn2tp\") pod \"maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g\" (UID: \"29e3ef22-7657-4fe5-803e-75a93978aced\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"
Oct 30 19:19:15.767647 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.767582    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29e3ef22-7657-4fe5-803e-75a93978aced-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g\" (UID: \"29e3ef22-7657-4fe5-803e-75a93978aced\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"
Oct 30 19:19:15.790140 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.789966    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29e3ef22-7657-4fe5-803e-75a93978aced-metrics-certs\") pod \"maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g\" (UID: \"29e3ef22-7657-4fe5-803e-75a93978aced\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"
Oct 30 19:19:15.792452 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:15.792421    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn2tp\" (UniqueName: \"kubernetes.io/projected/29e3ef22-7657-4fe5-803e-75a93978aced-kube-api-access-wn2tp\") pod \"maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g\" (UID: \"29e3ef22-7657-4fe5-803e-75a93978aced\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"
Oct 30 19:19:16.296824 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:16.296354    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29e3ef22-7657-4fe5-803e-75a93978aced-maas-api-tls\") pod \"maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g\" (UID: \"29e3ef22-7657-4fe5-803e-75a93978aced\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"
Oct 30 19:19:16.322571 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:16.322542    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29e3ef22-7657-4fe5-803e-75a93978aced-maas-api-tls\") pod \"maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g\" (UID: \"29e3ef22-7657-4fe5-803e-75a93978aced\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"
Oct 30 19:19:16.595806 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:16.592543    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"
Oct 30 19:19:17.015404 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:17.015307    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"]
Oct 30 19:19:17.027469 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:19:17.027432    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29e3ef22_7657_4fe5_803e_75a93978aced.slice/crio-cc866729d730621d51fe16ae3a237cb83861285ba6f21a683053d34da52a9878 WatchSource:0}: Error finding container cc866729d730621d51fe16ae3a237cb83861285ba6f21a683053d34da52a9878: Status 404 returned error can't find the container with id cc866729d730621d51fe16ae3a237cb83861285ba6f21a683053d34da52a9878
Oct 30 19:19:17.357798 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:17.357685    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g" event={"ID":"29e3ef22-7657-4fe5-803e-75a93978aced","Type":"ContainerStarted","Data":"cc866729d730621d51fe16ae3a237cb83861285ba6f21a683053d34da52a9878"}
Oct 30 19:19:18.427242 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:18.424277    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g" event={"ID":"29e3ef22-7657-4fe5-803e-75a93978aced","Type":"ContainerStarted","Data":"f4909b92b9fe474a0c65ad8d38fde96588fd10cf02c34c21aeb7ffea2b43afd5"}
Oct 30 19:19:18.427242 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:18.424343    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"
Oct 30 19:19:18.459190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:18.459102    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g" podStartSLOduration=2.925982768 podStartE2EDuration="3.459082678s" totalImagesPullingTime="533.09991ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:18.456864853 +0000 UTC m=+790.197855965" watchObservedRunningTime="2026-10-30 19:19:18.459082678 +0000 UTC m=+790.200073758"
Oct 30 19:19:20.338429 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:20.338129    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"
Oct 30 19:19:20.462610 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:20.462578    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h" event={"ID":"db66fd6a-afe9-47e7-815c-5c66c86081d8","Type":"ContainerStarted","Data":"37569f1588ec4102c1cb7a3ece5b5ac88d7408ec11a574ef809f03d51a2ac79a"}
Oct 30 19:19:20.463829 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:20.463479    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h"
Oct 30 19:19:20.466516 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:20.466281    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb" event={"ID":"1affe672-c981-43ad-bd2c-099648e4be4c","Type":"ContainerStarted","Data":"594a7a82458e83016d71647fd6c49048dfdc29051ba1b6e9d86a73e417db102f"}
Oct 30 19:19:20.466866 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:20.466846    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb"
Oct 30 19:19:20.473150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:20.469269    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd" event={"ID":"fb610784-5f32-446f-9096-f524917ae3c7","Type":"ContainerStarted","Data":"f780342c2e4527fa0abfbb49e2590ad11a48c7c1a9365ce718fc687df7c5afe2"}
Oct 30 19:19:20.473150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:20.469519    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd"
Oct 30 19:19:20.490154 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:20.487586    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h" podStartSLOduration=2.205642764 podStartE2EDuration="11.487567311s" totalImagesPullingTime="9.281924547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:20.485023508 +0000 UTC m=+792.226014586" watchObservedRunningTime="2026-10-30 19:19:20.487567311 +0000 UTC m=+792.228558390"
Oct 30 19:19:20.512931 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:20.512842    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd" podStartSLOduration=1.218504267 podStartE2EDuration="10.51282442s" totalImagesPullingTime="9.294320153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:20.504899777 +0000 UTC m=+792.245890853" watchObservedRunningTime="2026-10-30 19:19:20.51282442 +0000 UTC m=+792.253815503"
Oct 30 19:19:21.351494 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.351467    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"
Oct 30 19:19:21.351966 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.351911    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"
Oct 30 19:19:21.352506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.352483    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"
Oct 30 19:19:21.374777 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.374722    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb" podStartSLOduration=2.7219152920000003 podStartE2EDuration="12.374704162s" totalImagesPullingTime="9.65278887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:20.532380162 +0000 UTC m=+792.273371236" watchObservedRunningTime="2026-10-30 19:19:21.374704162 +0000 UTC m=+793.115695242"
Oct 30 19:19:21.481847 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.481821    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c" event={"ID":"4bac4098-9dc9-4305-b0ef-286d82cae682","Type":"ContainerStarted","Data":"152542d5a417fc063afb1acfbd5384dd4171ee535adedcde56e9254f330d32d1"}
Oct 30 19:19:21.482218 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.482041    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c"
Oct 30 19:19:21.483884 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.483861    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5" event={"ID":"5355e2dd-ba58-4602-bedc-44a2979762e1","Type":"ContainerStarted","Data":"77e7d5836186b50b3e4ebbe4a881cb0b05e5d69511a784fd468220fee8ef5e59"}
Oct 30 19:19:21.484399 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.484383    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5"
Oct 30 19:19:21.486066 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.486045    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d" event={"ID":"34a1d722-a8a4-45e0-a5e4-6306c70eb7bb","Type":"ContainerStarted","Data":"617aca740b732e2809707bb638e6595aba570469fbc6e61a7ea61ca149679b2f"}
Oct 30 19:19:21.486481 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.486463    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d"
Oct 30 19:19:21.488521 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.488499    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq" event={"ID":"7ac2f9eb-a945-483a-9c0a-7ad91b8fd618","Type":"ContainerStarted","Data":"e866f74e712d90cf48a0f818e0cdfdcbee70f49fab1722a1963d019b54907a8c"}
Oct 30 19:19:21.488683 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.488662    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq"
Oct 30 19:19:21.490935 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.490411    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8" event={"ID":"8c861815-5c5d-4a6a-9916-fcedf26bc4e0","Type":"ContainerStarted","Data":"0e9107cd32a24205c1758795c83c4b03b6a8ef3e9a365f5b5ed6c6af8d1b8158"}
Oct 30 19:19:21.490935 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.490886    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8"
Oct 30 19:19:21.492461 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.492438    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh" event={"ID":"3689eb31-fbba-4cbf-a355-7056ccec100b","Type":"ContainerStarted","Data":"7080939e88c720ab0317423c72842b71ecd72a8f429dfbef10c6fb5341766ff9"}
Oct 30 19:19:21.492619 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.492603    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh"
Oct 30 19:19:21.506355 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.506308    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c" podStartSLOduration=2.077942259 podStartE2EDuration="10.506291324s" totalImagesPullingTime="8.428349065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:21.502703601 +0000 UTC m=+793.243694680" watchObservedRunningTime="2026-10-30 19:19:21.506291324 +0000 UTC m=+793.247282405"
Oct 30 19:19:21.527235 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.527139    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8" podStartSLOduration=1.948502882 podStartE2EDuration="11.52712806s" totalImagesPullingTime="9.578625178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:21.526273263 +0000 UTC m=+793.267264342" watchObservedRunningTime="2026-10-30 19:19:21.52712806 +0000 UTC m=+793.268119154"
Oct 30 19:19:21.551509 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.551453    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5" podStartSLOduration=1.813058019 podStartE2EDuration="10.551437026s" totalImagesPullingTime="8.738379007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:21.547480027 +0000 UTC m=+793.288471106" watchObservedRunningTime="2026-10-30 19:19:21.551437026 +0000 UTC m=+793.292428109"
Oct 30 19:19:21.575776 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.575730    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh" podStartSLOduration=1.867020999 podStartE2EDuration="10.575717988s" totalImagesPullingTime="8.708696989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:21.574210146 +0000 UTC m=+793.315201226" watchObservedRunningTime="2026-10-30 19:19:21.575717988 +0000 UTC m=+793.316709066"
Oct 30 19:19:21.599354 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.599309    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d" podStartSLOduration=1.528172902 podStartE2EDuration="10.599298661s" totalImagesPullingTime="9.071125759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:21.597734343 +0000 UTC m=+793.338725422" watchObservedRunningTime="2026-10-30 19:19:21.599298661 +0000 UTC m=+793.340289741"
Oct 30 19:19:21.617331 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:21.617293    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq" podStartSLOduration=1.795265377 podStartE2EDuration="10.617281667s" totalImagesPullingTime="8.82201629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:21.615613248 +0000 UTC m=+793.356604329" watchObservedRunningTime="2026-10-30 19:19:21.617281667 +0000 UTC m=+793.358272746"
Oct 30 19:19:22.502271 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:22.502231    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88" event={"ID":"0806e982-fe3c-496d-95e2-c75d2b23dacf","Type":"ContainerStarted","Data":"aa01e1114c8827bc3a74044e3300d71e54db7d06bdf336f5daa00afa3db99880"}
Oct 30 19:19:22.502783 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:22.502716    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88"
Oct 30 19:19:22.524102 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:22.524057    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88" podStartSLOduration=2.712865227 podStartE2EDuration="10.524046182s" totalImagesPullingTime="7.811180955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:22.52026384 +0000 UTC m=+794.261254934" watchObservedRunningTime="2026-10-30 19:19:22.524046182 +0000 UTC m=+794.265037298"
Oct 30 19:19:24.442641 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:24.442612    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"
Oct 30 19:19:31.496022 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:31.495988    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd"
Oct 30 19:19:32.506136 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:32.506009    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq"
Oct 30 19:19:32.506136 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:32.506089    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh"
Oct 30 19:19:32.506630 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:32.506513    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c"
Oct 30 19:19:32.506630 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:32.506572    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb"
Oct 30 19:19:32.506630 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:32.506616    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5"
Oct 30 19:19:32.506893 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:32.506872    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h"
Oct 30 19:19:33.517189 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:33.512307    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d"
Oct 30 19:19:33.521360 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:33.521326    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8"
Oct 30 19:19:33.524121 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:19:33.524098    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88"
Oct 30 19:19:45.627429 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:19:45.627392    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:20:15.624927 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:20:15.624892    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:20:45.626844 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:20:45.626797    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:21:04.526425 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.526383    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"]
Oct 30 19:21:04.530596 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.530578    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.535470 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.535450    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-cfa4bb-openshift-default-dockercfg-jqxm7\""
Oct 30 19:21:04.551550 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.551518    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"]
Oct 30 19:21:04.725261 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.725226    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-data\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.725427 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.725288    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-podinfo\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.725427 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.725341    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-token\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.725427 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.725406    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-envoy\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.725531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.725493    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-workload-certs\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.725607 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.725579    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-credential-socket\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.725607 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.725599    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-workload-socket\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.725670 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.725624    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rm6fb\" (UniqueName: \"kubernetes.io/projected/e6dd71b4-1741-4da3-99f4-82121e585d89-kube-api-access-rm6fb\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.725721 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.725691    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6dd71b4-1741-4da3-99f4-82121e585d89-istiod-ca-cert\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.775658 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.775626    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"]
Oct 30 19:21:04.824312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.804458    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.832828    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-credential-socket\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.832870    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-workload-socket\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.832905    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-istio-envoy\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.832932    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rm6fb\" (UniqueName: \"kubernetes.io/projected/e6dd71b4-1741-4da3-99f4-82121e585d89-kube-api-access-rm6fb\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.832967    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6dd71b4-1741-4da3-99f4-82121e585d89-istiod-ca-cert\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.832998    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-credential-socket\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.833035    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-data\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.833071    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-podinfo\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.833094    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d898cf4d-a645-494a-a4af-88830acf24ab-istio-podinfo\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.833118    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-workload-socket\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.833150    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d898cf4d-a645-494a-a4af-88830acf24ab-istio-token\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.833191    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-token\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.833227    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-envoy\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.833254    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-workload-certs\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.833297    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-workload-certs\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.833341    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-istio-data\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.833378    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52jn9\" (UniqueName: \"kubernetes.io/projected/d898cf4d-a645-494a-a4af-88830acf24ab-kube-api-access-52jn9\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.833404    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d898cf4d-a645-494a-a4af-88830acf24ab-istiod-ca-cert\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.833846    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-credential-socket\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.834105    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-workload-socket\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.835064    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6dd71b4-1741-4da3-99f4-82121e585d89-istiod-ca-cert\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.835332    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-data\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.836739    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"]
Oct 30 19:21:04.837212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.837055    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-workload-certs\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.838875 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.838828    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-bce71f-openshift-default-dockercfg-w6thn\""
Oct 30 19:21:04.839089 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.839066    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-envoy\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.844112 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.844088    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-podinfo\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.848387 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.848367    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rm6fb\" (UniqueName: \"kubernetes.io/projected/e6dd71b4-1741-4da3-99f4-82121e585d89-kube-api-access-rm6fb\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.848972 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.848932    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-token\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:04.934380 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.934351    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-istio-envoy\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.934522 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.934401    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-credential-socket\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.934522 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.934436    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d898cf4d-a645-494a-a4af-88830acf24ab-istio-podinfo\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.934522 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.934456    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-workload-socket\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.934522 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.934481    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d898cf4d-a645-494a-a4af-88830acf24ab-istio-token\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.934522 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.934506    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-workload-certs\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.934755 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.934664    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-istio-data\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.934755 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.934718    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52jn9\" (UniqueName: \"kubernetes.io/projected/d898cf4d-a645-494a-a4af-88830acf24ab-kube-api-access-52jn9\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.934755 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.934749    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d898cf4d-a645-494a-a4af-88830acf24ab-istiod-ca-cert\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.934910 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.934855    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-workload-socket\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.934966 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.934915    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-workload-certs\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.935229 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.935206    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-credential-socket\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.935384 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.935232    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-istio-data\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.935531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.935511    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d898cf4d-a645-494a-a4af-88830acf24ab-istiod-ca-cert\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.937113 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.937093    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-istio-envoy\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.937323 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.937303    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d898cf4d-a645-494a-a4af-88830acf24ab-istio-podinfo\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.957283 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.957262    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52jn9\" (UniqueName: \"kubernetes.io/projected/d898cf4d-a645-494a-a4af-88830acf24ab-kube-api-access-52jn9\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:04.957366 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:04.957269    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d898cf4d-a645-494a-a4af-88830acf24ab-istio-token\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:05.136499 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:05.136465    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:05.144773 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:05.144751    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:05.463912 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:05.463885    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"]
Oct 30 19:21:05.465596 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:21:05.465574    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd898cf4d_a645_494a_a4af_88830acf24ab.slice/crio-3946a6e07356170ff4edb75dbd9dba30714d5b79017f7ed1aebc66b91270fb1b WatchSource:0}: Error finding container 3946a6e07356170ff4edb75dbd9dba30714d5b79017f7ed1aebc66b91270fb1b: Status 404 returned error can't find the container with id 3946a6e07356170ff4edb75dbd9dba30714d5b79017f7ed1aebc66b91270fb1b
Oct 30 19:21:05.467784 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:05.467747    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:05.467882 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:05.467818    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:05.467882 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:05.467850    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:05.480373 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:21:05.480349    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6dd71b4_1741_4da3_99f4_82121e585d89.slice/crio-f7d1f7a4a23f1f364b9c1fed6310a7f8eafc7dacc2565457f7cc24a18cf04c7b WatchSource:0}: Error finding container f7d1f7a4a23f1f364b9c1fed6310a7f8eafc7dacc2565457f7cc24a18cf04c7b: Status 404 returned error can't find the container with id f7d1f7a4a23f1f364b9c1fed6310a7f8eafc7dacc2565457f7cc24a18cf04c7b
Oct 30 19:21:05.482336 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:05.482312    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"]
Oct 30 19:21:05.482901 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:05.482872    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:05.482965 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:05.482933    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:05.482965 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:05.482961    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:06.251240 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:06.251199    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77" event={"ID":"d898cf4d-a645-494a-a4af-88830acf24ab","Type":"ContainerStarted","Data":"8eb10a7e60e00259dcee362b163edc8ea0d194016806354006be7861165e914a"}
Oct 30 19:21:06.251739 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:06.251281    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77" event={"ID":"d898cf4d-a645-494a-a4af-88830acf24ab","Type":"ContainerStarted","Data":"3946a6e07356170ff4edb75dbd9dba30714d5b79017f7ed1aebc66b91270fb1b"}
Oct 30 19:21:06.252651 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:06.252627    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp" event={"ID":"e6dd71b4-1741-4da3-99f4-82121e585d89","Type":"ContainerStarted","Data":"c37d23bb716ffaa764e13750ec186b6e3c3d43cdcd4bd6b8e493f90844e96c8c"}
Oct 30 19:21:06.252733 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:06.252668    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp" event={"ID":"e6dd71b4-1741-4da3-99f4-82121e585d89","Type":"ContainerStarted","Data":"f7d1f7a4a23f1f364b9c1fed6310a7f8eafc7dacc2565457f7cc24a18cf04c7b"}
Oct 30 19:21:06.291194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:06.282074    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77" podStartSLOduration=2.282054757 podStartE2EDuration="2.282054757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:06.279751205 +0000 UTC m=+898.020742287" watchObservedRunningTime="2026-10-30 19:21:06.282054757 +0000 UTC m=+898.023045838"
Oct 30 19:21:06.315388 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:06.315319    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp" podStartSLOduration=2.315298238 podStartE2EDuration="2.315298238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:06.313258644 +0000 UTC m=+898.054249726" watchObservedRunningTime="2026-10-30 19:21:06.315298238 +0000 UTC m=+898.056289317"
Oct 30 19:21:07.137767 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:07.137734    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:07.138770 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:07.138741    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 19:21:07.138843 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:07.138813    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77" podUID="d898cf4d-a645-494a-a4af-88830acf24ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 19:21:07.145857 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:07.145836    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:07.146977 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:07.146939    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 19:21:07.147024 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:07.146997    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp" podUID="e6dd71b4-1741-4da3-99f4-82121e585d89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 19:21:08.137378 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:08.137347    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 19:21:08.137813 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:08.137398    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77" podUID="d898cf4d-a645-494a-a4af-88830acf24ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 19:21:08.145124 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:08.145098    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 19:21:08.145241 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:08.145141    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp" podUID="e6dd71b4-1741-4da3-99f4-82121e585d89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 19:21:09.137439 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:09.137400    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 19:21:09.137868 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:09.137474    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77" podUID="d898cf4d-a645-494a-a4af-88830acf24ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 19:21:09.146078 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:09.146031    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 19:21:09.146224 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:09.146098    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp" podUID="e6dd71b4-1741-4da3-99f4-82121e585d89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 19:21:10.137205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:10.137158    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 19:21:10.137452 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:10.137425    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77" podUID="d898cf4d-a645-494a-a4af-88830acf24ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 19:21:10.145799 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:10.145767    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 19:21:10.145899 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:10.145809    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp" podUID="e6dd71b4-1741-4da3-99f4-82121e585d89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 19:21:11.136858 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:11.136822    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 19:21:11.137063 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:11.136881    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77" podUID="d898cf4d-a645-494a-a4af-88830acf24ab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 19:21:11.145016 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:11.144989    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Oct 30 19:21:11.145399 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:11.145053    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp" podUID="e6dd71b4-1741-4da3-99f4-82121e585d89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Oct 30 19:21:12.352786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:12.352753    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:12.360191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:12.354192    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:12.360191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:12.355549    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:21:12.363454 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:12.363404    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:12.365744 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:12.365721    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:12.367275 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:12.367256    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:21:15.625042 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:21:15.625002    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:21:27.376953 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.376917    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"]
Oct 30 19:21:27.381823 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.381802    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:27.395528 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.395413    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-2d5d53\"/\"openshift-service-ca.crt\""
Oct 30 19:21:27.395528 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.395465    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-2d5d53\"/\"default-dockercfg-8rj6n\""
Oct 30 19:21:27.395528 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.395495    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-2d5d53\"/\"kube-root-ca.crt\""
Oct 30 19:21:27.395804 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.395562    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-2d5d53\"/\"distinct-w5-2d5d53-kserve-self-signed-certs\""
Oct 30 19:21:27.397047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.397003    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"]
Oct 30 19:21:27.438188 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.438139    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-tls-certs\") pod \"distinct-w5-2d5d53-kserve-844959f975-gzsdz\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") " pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:27.438368 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.438233    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nxm6\" (UniqueName: \"kubernetes.io/projected/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-kube-api-access-7nxm6\") pod \"distinct-w5-2d5d53-kserve-844959f975-gzsdz\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") " pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:27.438368 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.438297    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-tmp-dir\") pod \"distinct-w5-2d5d53-kserve-844959f975-gzsdz\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") " pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:27.438368 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.438330    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-home\") pod \"distinct-w5-2d5d53-kserve-844959f975-gzsdz\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") " pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:27.438528 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.438500    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-dshm\") pod \"distinct-w5-2d5d53-kserve-844959f975-gzsdz\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") " pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:27.438589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.438550    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-model-cache\") pod \"distinct-w5-2d5d53-kserve-844959f975-gzsdz\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") " pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:27.540347 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.540257    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-tmp-dir\") pod \"distinct-w5-2d5d53-kserve-844959f975-gzsdz\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") " pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:27.553328 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.540608    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-home\") pod \"distinct-w5-2d5d53-kserve-844959f975-gzsdz\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") " pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:27.553624 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.549298    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-tmp-dir\") pod \"distinct-w5-2d5d53-kserve-844959f975-gzsdz\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") " pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:27.554433 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.549539    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-home\") pod \"distinct-w5-2d5d53-kserve-844959f975-gzsdz\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") " pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:27.554625 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.553812    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-dshm\") pod \"distinct-w5-2d5d53-kserve-844959f975-gzsdz\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") " pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:27.554805 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.554776    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-model-cache\") pod \"distinct-w5-2d5d53-kserve-844959f975-gzsdz\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") " pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:27.555482 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.555438    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-model-cache\") pod \"distinct-w5-2d5d53-kserve-844959f975-gzsdz\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") " pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:27.555647 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.555032    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-tls-certs\") pod \"distinct-w5-2d5d53-kserve-844959f975-gzsdz\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") " pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:27.564012 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.563642    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nxm6\" (UniqueName: \"kubernetes.io/projected/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-kube-api-access-7nxm6\") pod \"distinct-w5-2d5d53-kserve-844959f975-gzsdz\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") " pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:27.575029 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.574826    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-tls-certs\") pod \"distinct-w5-2d5d53-kserve-844959f975-gzsdz\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") " pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:27.607192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.603738    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-dshm\") pod \"distinct-w5-2d5d53-kserve-844959f975-gzsdz\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") " pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:27.615215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.612290    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nxm6\" (UniqueName: \"kubernetes.io/projected/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-kube-api-access-7nxm6\") pod \"distinct-w5-2d5d53-kserve-844959f975-gzsdz\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") " pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:27.705787 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:27.705407    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:28.016773 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:28.016746    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"]
Oct 30 19:21:28.018701 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:21:28.018672    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode57b4ffe_99fa_4948_a5d5_f4f0897bd0d0.slice/crio-eeb1b5c9a61584ae497225928909778a8a75a763dfbedc71559e869cd719f6fc WatchSource:0}: Error finding container eeb1b5c9a61584ae497225928909778a8a75a763dfbedc71559e869cd719f6fc: Status 404 returned error can't find the container with id eeb1b5c9a61584ae497225928909778a8a75a763dfbedc71559e869cd719f6fc
Oct 30 19:21:28.630570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:28.629693    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz" event={"ID":"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0","Type":"ContainerStarted","Data":"37cece85ba5947a81c70af51796893db4d7815e9c120f3d1303c8c570be603b7"}
Oct 30 19:21:28.630570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:28.630609    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz" event={"ID":"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0","Type":"ContainerStarted","Data":"eeb1b5c9a61584ae497225928909778a8a75a763dfbedc71559e869cd719f6fc"}
Oct 30 19:21:28.654700 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:28.654641    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz" podStartSLOduration=1.654596992 podStartE2EDuration="1.654596992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:28.651266488 +0000 UTC m=+920.392257568" watchObservedRunningTime="2026-10-30 19:21:28.654596992 +0000 UTC m=+920.395588070"
Oct 30 19:21:28.711600 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:28.710396    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:32.344313 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:32.344272    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-lm7vd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:21:32.344801 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:32.344362    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd" podUID="39af7415-b31d-4bc0-8a67-ec591643f5c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:21:34.663832 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.663795    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"]
Oct 30 19:21:34.674261 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.674224    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.681292 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.681267    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-27b5d5-openshift-default-dockercfg-djg8x\""
Oct 30 19:21:34.683816 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.683788    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"]
Oct 30 19:21:34.806326 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.806287    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-envoy\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.806504 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.806339    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-workload-certs\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.806504 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.806430    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-podinfo\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.806504 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.806485    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-credential-socket\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.806683 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.806578    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-token\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.806683 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.806607    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-workload-socket\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.806683 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.806676    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqt42\" (UniqueName: \"kubernetes.io/projected/75f42a71-342e-4c05-80c7-a1e7b6336984-kube-api-access-cqt42\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.806828 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.806721    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-data\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.806828 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.806751    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75f42a71-342e-4c05-80c7-a1e7b6336984-istiod-ca-cert\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.908062 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.908024    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-envoy\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.908062 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.908067    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-workload-certs\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.908344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.908091    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-podinfo\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.908344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.908122    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-credential-socket\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.908344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.908161    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-token\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.908344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.908225    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-workload-socket\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.908344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.908282    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqt42\" (UniqueName: \"kubernetes.io/projected/75f42a71-342e-4c05-80c7-a1e7b6336984-kube-api-access-cqt42\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.908344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.908328    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-data\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.908655 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.908366    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75f42a71-342e-4c05-80c7-a1e7b6336984-istiod-ca-cert\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.908709 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.908664    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-workload-socket\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.908965 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.908940    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-data\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.909049 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.908980    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-credential-socket\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.909190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.909150    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75f42a71-342e-4c05-80c7-a1e7b6336984-istiod-ca-cert\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.909264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.909192    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-workload-certs\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.910575 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.910549    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-envoy\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.911050 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.911029    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-podinfo\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.921091 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.921026    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-token\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.921644 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.921624    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqt42\" (UniqueName: \"kubernetes.io/projected/75f42a71-342e-4c05-80c7-a1e7b6336984-kube-api-access-cqt42\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:34.992570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:34.992545    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:35.235161 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.235089    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"]
Oct 30 19:21:35.240617 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.240593    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.243980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.243958    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-39b867-openshift-default-dockercfg-8z9sp\""
Oct 30 19:21:35.252704 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.252681    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"]
Oct 30 19:21:35.294736 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.282142    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"]
Oct 30 19:21:35.294736 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.286542    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:35.294736 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.286612    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:35.294736 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.286655    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:35.416112 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.416073    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-envoy\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.416360 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.416153    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0210704-8d10-4dad-8f61-7e028f2bbab8-istiod-ca-cert\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.416360 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.416243    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-token\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.416360 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.416287    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-credential-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.416360 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.416349    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-podinfo\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.416573 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.416379    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r45dm\" (UniqueName: \"kubernetes.io/projected/c0210704-8d10-4dad-8f61-7e028f2bbab8-kube-api-access-r45dm\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.416573 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.416442    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-workload-certs\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.416573 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.416461    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-workload-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.416573 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.416496    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-data\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.518414 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.518326    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-envoy\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.518711 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.518691    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0210704-8d10-4dad-8f61-7e028f2bbab8-istiod-ca-cert\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.518922 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.518905    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-token\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.519106 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.519092    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-credential-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.519258 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.519244    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-podinfo\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.519407 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.519381    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r45dm\" (UniqueName: \"kubernetes.io/projected/c0210704-8d10-4dad-8f61-7e028f2bbab8-kube-api-access-r45dm\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.519590 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.519576    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-workload-certs\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.519712 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.519697    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-workload-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.519876 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.519850    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-data\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.520431 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.519613    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-credential-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.520431 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.519997    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-workload-certs\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.520431 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.520231    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-workload-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.520431 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.519398    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0210704-8d10-4dad-8f61-7e028f2bbab8-istiod-ca-cert\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.520431 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.520266    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-data\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.521684 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.521629    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-envoy\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.523517 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.523491    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-podinfo\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.530792 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.530769    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-token\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.531045 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.531024    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r45dm\" (UniqueName: \"kubernetes.io/projected/c0210704-8d10-4dad-8f61-7e028f2bbab8-kube-api-access-r45dm\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.556544 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.556521    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:35.769436 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.769352    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6" event={"ID":"75f42a71-342e-4c05-80c7-a1e7b6336984","Type":"ContainerStarted","Data":"113ea6419af0ef9d096c81a5337ee8d5571cea01cce84b280f9969ab08ad43fb"}
Oct 30 19:21:35.769436 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.769414    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6" event={"ID":"75f42a71-342e-4c05-80c7-a1e7b6336984","Type":"ContainerStarted","Data":"28c17fc54c8cbe7d17b8c22b8d10993bb1a36fd591e4198d2060b82f782eb792"}
Oct 30 19:21:35.795239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.795188    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6" podStartSLOduration=1.795155295 podStartE2EDuration="1.795155295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:35.793628069 +0000 UTC m=+927.534619151" watchObservedRunningTime="2026-10-30 19:21:35.795155295 +0000 UTC m=+927.536146376"
Oct 30 19:21:35.850462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.850437    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"]
Oct 30 19:21:35.854199 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:21:35.854145    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0210704_8d10_4dad_8f61_7e028f2bbab8.slice/crio-b87110d1d94d5e99a8fc03b85f58cfa9fbed41caaa56ce51b80cc9a40a6b789e WatchSource:0}: Error finding container b87110d1d94d5e99a8fc03b85f58cfa9fbed41caaa56ce51b80cc9a40a6b789e: Status 404 returned error can't find the container with id b87110d1d94d5e99a8fc03b85f58cfa9fbed41caaa56ce51b80cc9a40a6b789e
Oct 30 19:21:35.856603 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.856572    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:35.856720 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.856637    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:35.856720 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.856668    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:35.992720 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:35.992679    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:36.848165 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:36.847638    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf" event={"ID":"c0210704-8d10-4dad-8f61-7e028f2bbab8","Type":"ContainerStarted","Data":"44a801cc581e21f3201f5f06290ac56ef7bbcf9d7acf76986d3c87e3628c09ee"}
Oct 30 19:21:36.848165 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:36.847703    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf" event={"ID":"c0210704-8d10-4dad-8f61-7e028f2bbab8","Type":"ContainerStarted","Data":"b87110d1d94d5e99a8fc03b85f58cfa9fbed41caaa56ce51b80cc9a40a6b789e"}
Oct 30 19:21:36.884193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:36.873263    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf" podStartSLOduration=1.873209197 podStartE2EDuration="1.873209197s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:36.871817106 +0000 UTC m=+928.612808187" watchObservedRunningTime="2026-10-30 19:21:36.873209197 +0000 UTC m=+928.614200278"
Oct 30 19:21:37.029193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:37.026941    4524 patch_prober.go:29] interesting pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:21:37.029193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:37.027036    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6" podUID="75f42a71-342e-4c05-80c7-a1e7b6336984" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 19:21:37.556767 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:37.556734    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:37.559057 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:37.559031    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 19:21:37.559222 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:37.559099    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf" podUID="c0210704-8d10-4dad-8f61-7e028f2bbab8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 19:21:37.715206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:37.711571    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:37.743657 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:37.743626    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:37.939531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:37.939488    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:21:38.018194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:38.017272    4524 patch_prober.go:29] interesting pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:21:38.018194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:38.017363    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6" podUID="75f42a71-342e-4c05-80c7-a1e7b6336984" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 19:21:38.558783 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:38.557785    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 19:21:38.558783 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:38.557868    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf" podUID="c0210704-8d10-4dad-8f61-7e028f2bbab8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 19:21:38.993987 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:38.993548    4524 patch_prober.go:29] interesting pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:21:38.993987 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:38.993622    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6" podUID="75f42a71-342e-4c05-80c7-a1e7b6336984" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 19:21:39.558199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:39.557780    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 19:21:39.558199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:39.557824    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf" podUID="c0210704-8d10-4dad-8f61-7e028f2bbab8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 19:21:39.998196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:39.997562    4524 patch_prober.go:29] interesting pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:21:39.998196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:39.997622    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6" podUID="75f42a71-342e-4c05-80c7-a1e7b6336984" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 19:21:40.557091 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:40.557055    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 19:21:40.557287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:40.557105    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf" podUID="c0210704-8d10-4dad-8f61-7e028f2bbab8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 19:21:40.995741 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:40.995689    4524 patch_prober.go:29] interesting pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:21:40.996085 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:40.996029    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6" podUID="75f42a71-342e-4c05-80c7-a1e7b6336984" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 19:21:41.557984 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:41.557939    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 19:21:41.558399 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:41.558008    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf" podUID="c0210704-8d10-4dad-8f61-7e028f2bbab8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 19:21:41.994001 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:41.993562    4524 patch_prober.go:29] interesting pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:21:41.994001 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:41.993622    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6" podUID="75f42a71-342e-4c05-80c7-a1e7b6336984" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 19:21:42.557871 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:42.557831    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Oct 30 19:21:42.558080 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:42.557897    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf" podUID="c0210704-8d10-4dad-8f61-7e028f2bbab8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Oct 30 19:21:42.998321 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:42.998285    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:42.998748 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:42.998722    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:42.999848 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:42.999824    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:21:43.831721 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:43.831688    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:43.832134 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:43.832102    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:43.832868 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:43.832848    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:21:45.625857 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:21:45.625820    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:21:49.614915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.614877    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"]
Oct 30 19:21:49.619430 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.619407    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.634039 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.634012    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"]
Oct 30 19:21:49.709554 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.709518    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"]
Oct 30 19:21:49.713511 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.713488    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.733688 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.733660    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"]
Oct 30 19:21:49.755539 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.755504    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-istio-data\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.755790 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.755753    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmxxq\" (UniqueName: \"kubernetes.io/projected/b0cd7cfe-e360-45e7-a17c-5067167d3afe-kube-api-access-rmxxq\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.755951 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.755833    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/849295c8-38ec-4630-95be-60a6200e4a20-istiod-ca-cert\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.756040 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.755945    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-data\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.756040 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.756020    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-workload-socket\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.756253 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.756057    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/849295c8-38ec-4630-95be-60a6200e4a20-istio-podinfo\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.756253 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.756089    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-workload-certs\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.756253 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.756143    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-credential-socket\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.756253 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.756204    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-envoy\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.756253 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.756238    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-workload-socket\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.756552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.756276    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-workload-certs\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.756552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.756305    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-token\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.756552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.756356    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istiod-ca-cert\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.756552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.756415    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-podinfo\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.756552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.756449    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx7vz\" (UniqueName: \"kubernetes.io/projected/849295c8-38ec-4630-95be-60a6200e4a20-kube-api-access-hx7vz\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.756552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.756475    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-credential-socket\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.756552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.756530    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-istio-envoy\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.756902 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.756554    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/849295c8-38ec-4630-95be-60a6200e4a20-istio-token\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.850231 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.850203    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"]
Oct 30 19:21:49.854815 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.854786    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:49.858635 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.858587    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/849295c8-38ec-4630-95be-60a6200e4a20-istio-podinfo\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.858784 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.858642    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-workload-certs\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.858784 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.858700    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-credential-socket\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.858784 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.858747    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-envoy\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.858955 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.858781    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-workload-socket\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.858955 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.858824    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-workload-certs\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.858955 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.858857    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-token\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.858955 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.858913    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istiod-ca-cert\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.859163 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.858976    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-podinfo\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.859163 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.859009    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx7vz\" (UniqueName: \"kubernetes.io/projected/849295c8-38ec-4630-95be-60a6200e4a20-kube-api-access-hx7vz\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.859163 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.859036    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-credential-socket\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.859163 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.859134    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-istio-envoy\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.859394 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.859163    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/849295c8-38ec-4630-95be-60a6200e4a20-istio-token\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.859394 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.859239    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-istio-data\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.859394 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.859266    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmxxq\" (UniqueName: \"kubernetes.io/projected/b0cd7cfe-e360-45e7-a17c-5067167d3afe-kube-api-access-rmxxq\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.859394 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.859309    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-workload-socket\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.859394 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.859324    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/849295c8-38ec-4630-95be-60a6200e4a20-istiod-ca-cert\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.859394 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.859369    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-data\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.859699 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.859411    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-workload-socket\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.859756 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.859731    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-workload-socket\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.859967 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.859946    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-workload-certs\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.860042 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.860024    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/849295c8-38ec-4630-95be-60a6200e4a20-istiod-ca-cert\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.860210 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.860192    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-credential-socket\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.860314 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.860294    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-workload-certs\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.861301 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.861274    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-data\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.861852 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.861828    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istiod-ca-cert\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.862070 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.862051    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-istio-data\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.863155 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.863131    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-istio-envoy\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.863961 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.863938    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-credential-socket\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.864475 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.864455    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/849295c8-38ec-4630-95be-60a6200e4a20-istio-podinfo\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.895957 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.869677    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-podinfo\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.895957 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.869850    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-envoy\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.895957 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.877814    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"]
Oct 30 19:21:49.895957 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.879245    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/849295c8-38ec-4630-95be-60a6200e4a20-istio-token\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.895957 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.880560    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-token\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.895957 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.885448    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx7vz\" (UniqueName: \"kubernetes.io/projected/849295c8-38ec-4630-95be-60a6200e4a20-kube-api-access-hx7vz\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.895957 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.886559    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmxxq\" (UniqueName: \"kubernetes.io/projected/b0cd7cfe-e360-45e7-a17c-5067167d3afe-kube-api-access-rmxxq\") pod \"e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") " pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:49.933590 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.933564    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:49.998281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.960727    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-workload-socket\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:49.998281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.960831    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-podinfo\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:49.998281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.960906    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-envoy\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:49.998281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.960953    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-credential-socket\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:49.998281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.960977    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-token\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:49.998281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.961012    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgb87\" (UniqueName: \"kubernetes.io/projected/77f56e59-c42b-4a38-a6c8-85f903e73922-kube-api-access-xgb87\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:49.998281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.961118    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-data\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:49.998281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.961148    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77f56e59-c42b-4a38-a6c8-85f903e73922-istiod-ca-cert\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:49.998281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:49.961196    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-workload-certs\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:50.023447 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.023406    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"]
Oct 30 19:21:50.032202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.031399    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:50.047625 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.047595    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.057017 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.056737    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-17e904-openshift-default-dockercfg-d5z8k\""
Oct 30 19:21:50.059436 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.059350    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"]
Oct 30 19:21:50.063914 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.063068    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77f56e59-c42b-4a38-a6c8-85f903e73922-istiod-ca-cert\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:50.063914 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.063113    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-workload-certs\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:50.063914 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.063200    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-workload-socket\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:50.063914 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.063617    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-workload-socket\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:50.063914 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.063865    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-workload-certs\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:50.063914 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.063897    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77f56e59-c42b-4a38-a6c8-85f903e73922-istiod-ca-cert\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:50.064312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.063945    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-podinfo\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:50.064312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.064020    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-envoy\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:50.064312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.064066    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-credential-socket\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:50.064312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.064090    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-token\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:50.064312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.064125    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgb87\" (UniqueName: \"kubernetes.io/projected/77f56e59-c42b-4a38-a6c8-85f903e73922-kube-api-access-xgb87\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:50.064312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.064252    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-data\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:50.064692 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.064565    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-data\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:50.065111 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.064806    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-credential-socket\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:50.070884 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.070160    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-podinfo\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:50.080738 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.080692    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-token\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:50.081256 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.081203    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-envoy\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:50.084156 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.084115    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgb87\" (UniqueName: \"kubernetes.io/projected/77f56e59-c42b-4a38-a6c8-85f903e73922-kube-api-access-xgb87\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:50.161510 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.161448    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"]
Oct 30 19:21:50.166143 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.165674    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-istio-envoy\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.166143 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.165723    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-istio-data\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.166143 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.165773    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-workload-socket\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.166143 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.165790    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpdj2\" (UniqueName: \"kubernetes.io/projected/a764618a-2358-46f2-bd29-a08c4b800a37-kube-api-access-fpdj2\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.166143 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.165807    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-workload-certs\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.166143 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.165829    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a764618a-2358-46f2-bd29-a08c4b800a37-istio-token\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.166143 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.165878    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a764618a-2358-46f2-bd29-a08c4b800a37-istiod-ca-cert\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.166143 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.165929    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a764618a-2358-46f2-bd29-a08c4b800a37-istio-podinfo\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.166143 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.165959    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-credential-socket\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.167096 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.167075    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.188738 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.187610    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"]
Oct 30 19:21:50.188738 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.188200    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-ad23c7-openshift-default-dockercfg-b7xt7\""
Oct 30 19:21:50.230723 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.230692    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:50.266971 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.266906    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-workload-socket\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.266971 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.266953    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpdj2\" (UniqueName: \"kubernetes.io/projected/a764618a-2358-46f2-bd29-a08c4b800a37-kube-api-access-fpdj2\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.267127 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.266984    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-workload-certs\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.267127 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.267024    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a764618a-2358-46f2-bd29-a08c4b800a37-istio-token\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.267127 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.267069    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54369e69-c268-48e4-80cd-84021afa2d10-istiod-ca-cert\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.267302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.267134    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a764618a-2358-46f2-bd29-a08c4b800a37-istiod-ca-cert\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.267302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.267188    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-workload-socket\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.267302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.267223    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-istio-envoy\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.267302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.267257    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-workload-certs\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.267302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.267298    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-istio-data\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.267543 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.267352    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a764618a-2358-46f2-bd29-a08c4b800a37-istio-podinfo\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.267543 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.267376    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54369e69-c268-48e4-80cd-84021afa2d10-istio-token\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.267543 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.267432    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-credential-socket\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.267543 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.267489    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-istio-envoy\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.267543 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.267518    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-credential-socket\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.267800 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.267574    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54369e69-c268-48e4-80cd-84021afa2d10-istio-podinfo\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.267800 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.267603    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-istio-data\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.267800 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.267651    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl6nj\" (UniqueName: \"kubernetes.io/projected/54369e69-c268-48e4-80cd-84021afa2d10-kube-api-access-pl6nj\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.268191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.268146    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-workload-socket\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.268833 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.268798    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-workload-certs\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.268932 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.268878    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-istio-data\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.269317 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.269287    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-credential-socket\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.269622 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.269557    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a764618a-2358-46f2-bd29-a08c4b800a37-istiod-ca-cert\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.271700 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.271678    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-istio-envoy\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.271813 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.271706    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a764618a-2358-46f2-bd29-a08c4b800a37-istio-podinfo\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.280268 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.280242    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpdj2\" (UniqueName: \"kubernetes.io/projected/a764618a-2358-46f2-bd29-a08c4b800a37-kube-api-access-fpdj2\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.280666 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.280646    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a764618a-2358-46f2-bd29-a08c4b800a37-istio-token\") pod \"e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") " pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.289748 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:21:50.289699    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod849295c8_38ec_4630_95be_60a6200e4a20.slice/crio-d77e85499014fe0c25578b3a8baa2a03e74502afd15f87fa5ced101313982942 WatchSource:0}: Error finding container d77e85499014fe0c25578b3a8baa2a03e74502afd15f87fa5ced101313982942: Status 404 returned error can't find the container with id d77e85499014fe0c25578b3a8baa2a03e74502afd15f87fa5ced101313982942
Oct 30 19:21:50.294745 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.294572    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:50.294745 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.294687    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:50.294745 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.294739    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:50.296375 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.296108    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"]
Oct 30 19:21:50.360809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.360586    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"]
Oct 30 19:21:50.361407 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:21:50.361361    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0cd7cfe_e360_45e7_a17c_5067167d3afe.slice/crio-d0b79ffe679314c34c76ddbb0a288acb41e681c4acec4b0aaf014d6b7ba6b19a WatchSource:0}: Error finding container d0b79ffe679314c34c76ddbb0a288acb41e681c4acec4b0aaf014d6b7ba6b19a: Status 404 returned error can't find the container with id d0b79ffe679314c34c76ddbb0a288acb41e681c4acec4b0aaf014d6b7ba6b19a
Oct 30 19:21:50.365303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.364358    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:50.365303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.364434    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:50.365303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.364466    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:50.376878 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.376711    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54369e69-c268-48e4-80cd-84021afa2d10-istio-token\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.376878 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.376798    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-credential-socket\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.376878 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.376841    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54369e69-c268-48e4-80cd-84021afa2d10-istio-podinfo\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.377569 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.377210    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-credential-socket\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.377828 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.377784    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl6nj\" (UniqueName: \"kubernetes.io/projected/54369e69-c268-48e4-80cd-84021afa2d10-kube-api-access-pl6nj\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.378558 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.377979    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54369e69-c268-48e4-80cd-84021afa2d10-istiod-ca-cert\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.378558 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.378037    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-workload-socket\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.378558 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.378054    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-istio-envoy\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.378558 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.378074    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-workload-certs\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.378558 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.378097    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-istio-data\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.378558 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.378445    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-istio-data\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.378864 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.378622    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-workload-certs\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.379370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.379308    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-workload-socket\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.379645 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.379608    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54369e69-c268-48e4-80cd-84021afa2d10-istiod-ca-cert\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.380629 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.380572    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54369e69-c268-48e4-80cd-84021afa2d10-istio-podinfo\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.382444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.382425    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:50.385779 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.385750    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-istio-envoy\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.394954 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.394929    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54369e69-c268-48e4-80cd-84021afa2d10-istio-token\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.402918 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.402879    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl6nj\" (UniqueName: \"kubernetes.io/projected/54369e69-c268-48e4-80cd-84021afa2d10-kube-api-access-pl6nj\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.502633 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.502598    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:50.550711 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.550682    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"]
Oct 30 19:21:50.563213 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.559961    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:50.563213 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.560046    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:50.563213 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.560089    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:50.724045 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.723902    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"]
Oct 30 19:21:50.734792 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.734563    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:50.735094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.734865    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:50.735300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.735030    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:50.934229 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.933314    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"]
Oct 30 19:21:50.939302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.938461    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:50.939302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.938533    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:50.939302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:50.938576    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:21:51.035491 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.035461    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr" event={"ID":"b0cd7cfe-e360-45e7-a17c-5067167d3afe","Type":"ContainerStarted","Data":"c8cee943882a26ddc3a76db9577794b618c92395baa26fc87529fe57faeffa99"}
Oct 30 19:21:51.035631 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.035521    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr" event={"ID":"b0cd7cfe-e360-45e7-a17c-5067167d3afe","Type":"ContainerStarted","Data":"d0b79ffe679314c34c76ddbb0a288acb41e681c4acec4b0aaf014d6b7ba6b19a"}
Oct 30 19:21:51.038282 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.038242    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb" event={"ID":"849295c8-38ec-4630-95be-60a6200e4a20","Type":"ContainerStarted","Data":"2b23122467660529993982cfcf2c7902b918d4514263538d764e1dc44cffc662"}
Oct 30 19:21:51.038390 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.038306    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb" event={"ID":"849295c8-38ec-4630-95be-60a6200e4a20","Type":"ContainerStarted","Data":"d77e85499014fe0c25578b3a8baa2a03e74502afd15f87fa5ced101313982942"}
Oct 30 19:21:51.041066 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.041013    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422" event={"ID":"54369e69-c268-48e4-80cd-84021afa2d10","Type":"ContainerStarted","Data":"8dee0d42d99f6c793de8528031e397426c8a76d35d94a735eb7af0947bf9a0cb"}
Oct 30 19:21:51.042349 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.042288    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:21:51.043703 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.043659    4524 patch_prober.go:29] interesting pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:21:51.043819 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.043729    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr" podUID="b0cd7cfe-e360-45e7-a17c-5067167d3afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:21:51.044765 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.044742    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l" event={"ID":"a764618a-2358-46f2-bd29-a08c4b800a37","Type":"ContainerStarted","Data":"aebe670b587c00acb61825e57aff17738062ef418d682ef03646d08d34fb8754"}
Oct 30 19:21:51.044865 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.044786    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l" event={"ID":"a764618a-2358-46f2-bd29-a08c4b800a37","Type":"ContainerStarted","Data":"3b5b3f25ffdd2c00f6fbf38e21c519ef0f29605ce61e3ff7d5c59517a213cb30"}
Oct 30 19:21:51.048019 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.047996    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj" event={"ID":"77f56e59-c42b-4a38-a6c8-85f903e73922","Type":"ContainerStarted","Data":"e3289a960ec230768eb64b2f42165e810fac1ee7b78a83c256020e17d7c73943"}
Oct 30 19:21:51.048157 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.048036    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj" event={"ID":"77f56e59-c42b-4a38-a6c8-85f903e73922","Type":"ContainerStarted","Data":"9df6157daa8c8761fbfd71db1d6f2fb38360ac0d1d7f62e2c7b4d20ea33dd72e"}
Oct 30 19:21:51.160013 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.159951    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr" podStartSLOduration=2.159931038 podStartE2EDuration="2.159931038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:51.103254301 +0000 UTC m=+942.844245382" watchObservedRunningTime="2026-10-30 19:21:51.159931038 +0000 UTC m=+942.900922120"
Oct 30 19:21:51.208199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.206507    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj" podStartSLOduration=2.206484503 podStartE2EDuration="2.206484503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:51.166224676 +0000 UTC m=+942.907215779" watchObservedRunningTime="2026-10-30 19:21:51.206484503 +0000 UTC m=+942.947475585"
Oct 30 19:21:51.236270 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.235683    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:21:51.236472 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.236271    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:21:51.236472 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.236320    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj" podUID="77f56e59-c42b-4a38-a6c8-85f903e73922" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:21:51.247436 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.247384    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb" podStartSLOduration=2.247361922 podStartE2EDuration="2.247361922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:51.215741199 +0000 UTC m=+942.956732279" watchObservedRunningTime="2026-10-30 19:21:51.247361922 +0000 UTC m=+942.988353005"
Oct 30 19:21:51.390565 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.390527    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:21:51.400490 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.400463    4524 patch_prober.go:29] interesting pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:21:51.400714 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.400694    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l" podUID="a764618a-2358-46f2-bd29-a08c4b800a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:21:51.946694 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.944957    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:21:51.946694 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.945022    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb" podUID="849295c8-38ec-4630-95be-60a6200e4a20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:21:51.946694 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:51.946625    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:21:52.043142 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:52.043106    4524 patch_prober.go:29] interesting pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:21:52.043461 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:52.043428    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr" podUID="b0cd7cfe-e360-45e7-a17c-5067167d3afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:21:52.196423 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:52.189403    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422" event={"ID":"54369e69-c268-48e4-80cd-84021afa2d10","Type":"ContainerStarted","Data":"946d7ea41696a16e1430476a4db868f5344e2d86d40bbc92dc5691a0736fab9b"}
Oct 30 19:21:52.264355 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:52.254960    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:21:52.264355 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:52.255029    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj" podUID="77f56e59-c42b-4a38-a6c8-85f903e73922" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:21:52.391070 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:52.391034    4524 patch_prober.go:29] interesting pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:21:52.391398 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:52.391342    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l" podUID="a764618a-2358-46f2-bd29-a08c4b800a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:21:52.427486 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:52.410007    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l" podStartSLOduration=3.409981106 podStartE2EDuration="3.409981106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:51.253738956 +0000 UTC m=+942.994730038" watchObservedRunningTime="2026-10-30 19:21:52.409981106 +0000 UTC m=+944.150972191"
Oct 30 19:21:52.533649 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:52.532065    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:21:52.533649 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:52.532161    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422" podUID="54369e69-c268-48e4-80cd-84021afa2d10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 19:21:52.628190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:52.621713    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:21:52.940096 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:52.939614    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:21:52.940096 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:52.939692    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb" podUID="849295c8-38ec-4630-95be-60a6200e4a20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:21:53.145202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:53.142889    4524 patch_prober.go:29] interesting pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:21:53.145202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:53.143009    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr" podUID="b0cd7cfe-e360-45e7-a17c-5067167d3afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:21:53.242269 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:53.235040    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:21:53.365518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:53.365476    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj" podUID="77f56e59-c42b-4a38-a6c8-85f903e73922" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:21:53.396324 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:53.396290    4524 patch_prober.go:29] interesting pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:21:53.396590 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:53.396566    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l" podUID="a764618a-2358-46f2-bd29-a08c4b800a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:21:53.505371 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:53.505267    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:21:53.505371 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:53.505333    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422" podUID="54369e69-c268-48e4-80cd-84021afa2d10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 19:21:53.938617 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:53.938577    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:21:53.938903 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:53.938870    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb" podUID="849295c8-38ec-4630-95be-60a6200e4a20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:21:54.048574 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:54.048536    4524 patch_prober.go:29] interesting pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:21:54.048812 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:54.048784    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr" podUID="b0cd7cfe-e360-45e7-a17c-5067167d3afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:21:54.236023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:54.235517    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:21:54.236023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:54.235578    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj" podUID="77f56e59-c42b-4a38-a6c8-85f903e73922" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:21:54.400223 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:54.389526    4524 patch_prober.go:29] interesting pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:21:54.400223 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:54.389587    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l" podUID="a764618a-2358-46f2-bd29-a08c4b800a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:21:54.504107 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:54.503597    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:21:54.504107 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:54.503661    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422" podUID="54369e69-c268-48e4-80cd-84021afa2d10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 19:21:54.937197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:54.934658    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:21:54.937197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:54.934731    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb" podUID="849295c8-38ec-4630-95be-60a6200e4a20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:21:55.042578 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:55.042542    4524 patch_prober.go:29] interesting pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:21:55.042868 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:55.042840    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr" podUID="b0cd7cfe-e360-45e7-a17c-5067167d3afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:21:55.234664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:55.234566    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:21:55.234914 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:55.234879    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj" podUID="77f56e59-c42b-4a38-a6c8-85f903e73922" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:21:55.392115 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:55.392081    4524 patch_prober.go:29] interesting pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:21:55.392681 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:55.392651    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l" podUID="a764618a-2358-46f2-bd29-a08c4b800a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:21:55.505249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:55.504541    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:21:55.505249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:55.504613    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422" podUID="54369e69-c268-48e4-80cd-84021afa2d10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 19:21:55.935576 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:55.935540    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:21:55.935852 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:55.935822    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb" podUID="849295c8-38ec-4630-95be-60a6200e4a20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:21:56.043196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:56.042571    4524 patch_prober.go:29] interesting pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:21:56.043196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:56.042637    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr" podUID="b0cd7cfe-e360-45e7-a17c-5067167d3afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:21:56.235764 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:56.235665    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:21:56.236065 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:56.236007    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj" podUID="77f56e59-c42b-4a38-a6c8-85f903e73922" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:21:56.406196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:56.401270    4524 patch_prober.go:29] interesting pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:21:56.406196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:56.401330    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l" podUID="a764618a-2358-46f2-bd29-a08c4b800a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:21:56.505844 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:56.504565    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:21:56.505844 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:56.504625    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422" podUID="54369e69-c268-48e4-80cd-84021afa2d10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 19:21:56.956203 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:56.955592    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:21:56.956203 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:56.955668    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb" podUID="849295c8-38ec-4630-95be-60a6200e4a20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:21:57.041599 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:57.041561    4524 patch_prober.go:29] interesting pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:21:57.041842 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:57.041810    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr" podUID="b0cd7cfe-e360-45e7-a17c-5067167d3afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:21:57.263758 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:57.263272    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:21:57.263758 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:57.263333    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj" podUID="77f56e59-c42b-4a38-a6c8-85f903e73922" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:21:57.388918 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:57.388510    4524 patch_prober.go:29] interesting pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:21:57.388918 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:57.388579    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l" podUID="a764618a-2358-46f2-bd29-a08c4b800a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:21:57.514256 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:57.512570    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:21:57.514256 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:57.512644    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422" podUID="54369e69-c268-48e4-80cd-84021afa2d10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 19:21:57.941196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:57.940747    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:21:57.941196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:57.940807    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb" podUID="849295c8-38ec-4630-95be-60a6200e4a20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:21:58.054196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:58.044789    4524 patch_prober.go:29] interesting pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:21:58.054196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:58.044852    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr" podUID="b0cd7cfe-e360-45e7-a17c-5067167d3afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:21:58.233718 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:58.233622    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:21:58.234000 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:58.233960    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj" podUID="77f56e59-c42b-4a38-a6c8-85f903e73922" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:21:58.390584 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:58.390549    4524 patch_prober.go:29] interesting pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:21:58.390861 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:58.390835    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l" podUID="a764618a-2358-46f2-bd29-a08c4b800a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:21:58.524247 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:58.523417    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:21:58.524247 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:58.523477    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422" podUID="54369e69-c268-48e4-80cd-84021afa2d10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 19:21:58.982444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:58.982409    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:21:58.982751 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:58.982725    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb" podUID="849295c8-38ec-4630-95be-60a6200e4a20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:21:59.092196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:59.083758    4524 patch_prober.go:29] interesting pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:21:59.092196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:59.083828    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr" podUID="b0cd7cfe-e360-45e7-a17c-5067167d3afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:21:59.233260 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:59.231542    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:21:59.233260 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:59.231603    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj" podUID="77f56e59-c42b-4a38-a6c8-85f903e73922" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:21:59.570250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:59.569455    4524 patch_prober.go:29] interesting pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:21:59.570250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:59.569519    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l" podUID="a764618a-2358-46f2-bd29-a08c4b800a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:21:59.633195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:59.572339    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:21:59.633195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:59.572388    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422" podUID="54369e69-c268-48e4-80cd-84021afa2d10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 19:21:59.937195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:59.936816    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:21:59.937195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:21:59.936886    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb" podUID="849295c8-38ec-4630-95be-60a6200e4a20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:22:00.042956 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:00.042920    4524 patch_prober.go:29] interesting pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:22:00.043240 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:00.043208    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr" podUID="b0cd7cfe-e360-45e7-a17c-5067167d3afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:22:00.241356 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:00.241268    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:22:00.241641 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:00.241611    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj" podUID="77f56e59-c42b-4a38-a6c8-85f903e73922" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:22:00.388616 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:00.388574    4524 patch_prober.go:29] interesting pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:22:00.388892 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:00.388862    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l" podUID="a764618a-2358-46f2-bd29-a08c4b800a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:22:00.503930 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:00.503568    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:22:00.503930 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:00.503637    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422" podUID="54369e69-c268-48e4-80cd-84021afa2d10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 19:22:00.947552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:00.947518    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:22:00.948600 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:00.948124    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb" podUID="849295c8-38ec-4630-95be-60a6200e4a20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:22:01.116488 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:01.095325    4524 patch_prober.go:29] interesting pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:22:01.116488 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:01.095383    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr" podUID="b0cd7cfe-e360-45e7-a17c-5067167d3afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:22:01.238861 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:01.238777    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:22:01.247281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:01.247237    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj" podUID="77f56e59-c42b-4a38-a6c8-85f903e73922" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:22:01.458196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:01.457725    4524 patch_prober.go:29] interesting pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:22:01.458196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:01.457799    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l" podUID="a764618a-2358-46f2-bd29-a08c4b800a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:22:01.570254 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:01.505139    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:22:01.570552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:01.570512    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422" podUID="54369e69-c268-48e4-80cd-84021afa2d10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 19:22:01.945195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:01.944687    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body=
Oct 30 19:22:01.945195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:01.944750    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb" podUID="849295c8-38ec-4630-95be-60a6200e4a20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused"
Oct 30 19:22:02.046819 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:02.046780    4524 patch_prober.go:29] interesting pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:22:02.047456 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:02.047422    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr" podUID="b0cd7cfe-e360-45e7-a17c-5067167d3afe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused"
Oct 30 19:22:02.231696 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:02.231600    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused" start-of-body=
Oct 30 19:22:02.231962 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:02.231927    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj" podUID="77f56e59-c42b-4a38-a6c8-85f903e73922" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.78:15021/healthz/ready\": dial tcp 10.133.0.78:15021: connect: connection refused"
Oct 30 19:22:02.392610 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:02.392571    4524 patch_prober.go:29] interesting pod/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 19:22:02.392895 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:02.392863    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l" podUID="a764618a-2358-46f2-bd29-a08c4b800a37" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 19:22:02.508246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:02.505545    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:22:02.508246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:02.505603    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422" podUID="54369e69-c268-48e4-80cd-84021afa2d10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 19:22:03.503309 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:03.503277    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:22:03.505579 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:03.505559    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:22:03.506251 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:03.506229    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:22:03.506392 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:03.506371    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422" podUID="54369e69-c268-48e4-80cd-84021afa2d10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 19:22:03.515450 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:03.515426    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:22:03.538055 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:03.538025    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:22:03.539664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:03.539558    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:22:03.548194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:03.544553    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422" podStartSLOduration=13.544531437 podStartE2EDuration="13.544531437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:52.254594147 +0000 UTC m=+943.995585226" watchObservedRunningTime="2026-10-30 19:22:03.544531437 +0000 UTC m=+955.285522545"
Oct 30 19:22:03.579570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:03.579541    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:22:03.756873 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:03.756790    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:22:03.758543 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:03.758520    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:22:03.762328 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:03.762306    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:22:03.814521 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:03.814490    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:22:03.817626 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:03.817604    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:22:03.821030 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:03.821016    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:22:05.065820 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.065783    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"]
Oct 30 19:22:05.083608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.083577    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.093990 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.093960    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-2d5d53-openshift-default-dockercfg-mbh26\""
Oct 30 19:22:05.163376 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.163314    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"]
Oct 30 19:22:05.260911 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.260881    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e53dd061-8aa9-4ace-9140-aea8a7375c20-istiod-ca-cert\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.261215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.261191    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-data\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.261293 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.261223    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-token\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.261293 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.261279    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-podinfo\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.261428 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.261417    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-workload-certs\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.261469 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.261437    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-envoy\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.261504 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.261474    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-workload-socket\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.261553 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.261542    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wk2v\" (UniqueName: \"kubernetes.io/projected/e53dd061-8aa9-4ace-9140-aea8a7375c20-kube-api-access-5wk2v\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.261605 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.261595    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-credential-socket\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.362904 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.362857    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e53dd061-8aa9-4ace-9140-aea8a7375c20-istiod-ca-cert\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.363073 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.362918    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-data\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.363073 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.362935    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-token\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.363073 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.362973    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-podinfo\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.363073 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.363055    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-workload-certs\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.363073 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.363071    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-envoy\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.363406 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.363104    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-workload-socket\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.363406 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.363150    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wk2v\" (UniqueName: \"kubernetes.io/projected/e53dd061-8aa9-4ace-9140-aea8a7375c20-kube-api-access-5wk2v\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.363406 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.363234    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-credential-socket\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.363983 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.363958    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e53dd061-8aa9-4ace-9140-aea8a7375c20-istiod-ca-cert\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.364061 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.364024    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-workload-certs\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.364295 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.364274    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-data\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.364982 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.364961    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-credential-socket\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.365270 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.365236    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-workload-socket\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.376644 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.376618    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-podinfo\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.381496 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.381471    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wk2v\" (UniqueName: \"kubernetes.io/projected/e53dd061-8aa9-4ace-9140-aea8a7375c20-kube-api-access-5wk2v\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.384327 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.382095    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-token\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.418653 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.418601    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-envoy\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.475808 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.474486    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:05.503462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.503432    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:22:05.578466 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.577511    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:22:05.578466 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.577572    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422" podUID="54369e69-c268-48e4-80cd-84021afa2d10" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:22:05.903537 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:22:05.903487    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode53dd061_8aa9_4ace_9140_aea8a7375c20.slice/crio-2e07d8c7624dfb06fcd06adb70f4809c834cd6e6bb45f711a8720fbfae241596 WatchSource:0}: Error finding container 2e07d8c7624dfb06fcd06adb70f4809c834cd6e6bb45f711a8720fbfae241596: Status 404 returned error can't find the container with id 2e07d8c7624dfb06fcd06adb70f4809c834cd6e6bb45f711a8720fbfae241596
Oct 30 19:22:05.904690 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.904666    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"]
Oct 30 19:22:05.906430 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.906397    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:22:05.906527 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.906458    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:22:05.906527 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:05.906486    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:22:06.080578 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:06.080541    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:22:06.090273 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:06.090162    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd" event={"ID":"e53dd061-8aa9-4ace-9140-aea8a7375c20","Type":"ContainerStarted","Data":"64fa6bef4c93bcee559a775b47195fcbcdf08108406e3c784e0391d901a5af49"}
Oct 30 19:22:06.090273 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:06.090235    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd" event={"ID":"e53dd061-8aa9-4ace-9140-aea8a7375c20","Type":"ContainerStarted","Data":"2e07d8c7624dfb06fcd06adb70f4809c834cd6e6bb45f711a8720fbfae241596"}
Oct 30 19:22:06.093310 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:06.093290    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:22:06.160321 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:06.151674    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd" podStartSLOduration=1.151655827 podStartE2EDuration="1.151655827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:06.150069992 +0000 UTC m=+957.891061084" watchObservedRunningTime="2026-10-30 19:22:06.151655827 +0000 UTC m=+957.892646907"
Oct 30 19:22:06.501115 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:06.500986    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:06.510689 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:06.510311    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:22:06.510689 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:06.510383    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd" podUID="e53dd061-8aa9-4ace-9140-aea8a7375c20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 19:22:07.498628 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:07.498591    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:22:07.499013 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:07.498649    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd" podUID="e53dd061-8aa9-4ace-9140-aea8a7375c20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 19:22:08.499343 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:08.499309    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:22:08.499947 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:08.499915    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd" podUID="e53dd061-8aa9-4ace-9140-aea8a7375c20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 19:22:09.498617 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:09.498579    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:22:09.498894 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:09.498868    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd" podUID="e53dd061-8aa9-4ace-9140-aea8a7375c20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 19:22:10.498604 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:10.498564    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:22:10.499236 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:10.499199    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd" podUID="e53dd061-8aa9-4ace-9140-aea8a7375c20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 19:22:11.499007 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:11.498975    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:22:11.499430 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:11.499030    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd" podUID="e53dd061-8aa9-4ace-9140-aea8a7375c20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 19:22:12.498995 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:12.498964    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:22:12.499182 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:12.499016    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd" podUID="e53dd061-8aa9-4ace-9140-aea8a7375c20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused"
Oct 30 19:22:13.502952 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:13.502922    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:13.503462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:13.503404    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:13.504139 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:13.504109    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:22:15.625662 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:22:15.625613    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:22:45.638889 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:22:45.638771    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:22:50.134558 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.134518    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"]
Oct 30 19:22:50.140063 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.140041    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.152315 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.152290    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"]
Oct 30 19:22:50.277068 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.277034    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvbzd\" (UniqueName: \"kubernetes.io/projected/68953aed-0987-40c8-932c-fa4591f71831-kube-api-access-tvbzd\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.277267 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.277097    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-workload-socket\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.277267 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.277242    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-istio-envoy\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.277380 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.277304    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-istio-data\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.277449 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.277427    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68953aed-0987-40c8-932c-fa4591f71831-istio-podinfo\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.277509 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.277493    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-credential-socket\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.277566 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.277532    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68953aed-0987-40c8-932c-fa4591f71831-istio-token\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.277614 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.277562    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-workload-certs\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.277738 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.277712    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68953aed-0987-40c8-932c-fa4591f71831-istiod-ca-cert\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.378908 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.378871    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68953aed-0987-40c8-932c-fa4591f71831-istiod-ca-cert\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.379067 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.379014    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvbzd\" (UniqueName: \"kubernetes.io/projected/68953aed-0987-40c8-932c-fa4591f71831-kube-api-access-tvbzd\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.379110 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.379063    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-workload-socket\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.379263 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.379242    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-istio-envoy\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.379320 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.379297    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-istio-data\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.379373 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.379327    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68953aed-0987-40c8-932c-fa4591f71831-istio-podinfo\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.379373 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.379349    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-credential-socket\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.379531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.379376    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68953aed-0987-40c8-932c-fa4591f71831-istio-token\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.379531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.379407    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-workload-certs\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.379531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.379460    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-workload-socket\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.379683 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.379662    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-credential-socket\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.379740 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.379687    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68953aed-0987-40c8-932c-fa4591f71831-istiod-ca-cert\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.379834 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.379816    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-workload-certs\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.379997 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.379976    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-istio-data\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.381651 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.381626    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-istio-envoy\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.381851 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.381832    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68953aed-0987-40c8-932c-fa4591f71831-istio-podinfo\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.396951 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.396897    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvbzd\" (UniqueName: \"kubernetes.io/projected/68953aed-0987-40c8-932c-fa4591f71831-kube-api-access-tvbzd\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.407483 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.407463    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68953aed-0987-40c8-932c-fa4591f71831-istio-token\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.445222 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.445191    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"]
Oct 30 19:22:50.451709 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.451684    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.456646 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.456623    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:50.521665 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.521588    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"]
Oct 30 19:22:50.526463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.526433    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"]
Oct 30 19:22:50.537756 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.537579    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.595193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.581765    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"]
Oct 30 19:22:50.595193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.582666    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w8nh\" (UniqueName: \"kubernetes.io/projected/a4fd9180-f4aa-4986-b2f0-955296fe17ee-kube-api-access-2w8nh\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.595193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.582792    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-workload-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.595193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.582875    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istiod-ca-cert\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.595193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.582975    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-workload-certs\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.595193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.583133    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-envoy\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.595193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.583212    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-credential-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.595193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.583311    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-data\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.595193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.583403    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-podinfo\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.595193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.583502    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-token\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.685230 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.685124    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-workload-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.685230 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.685164    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-envoy\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.685230 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.685203    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istiod-ca-cert\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.685230 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.685237    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-workload-certs\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.685230 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.685253    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-podinfo\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.685664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.685321    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-envoy\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.685664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.685356    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-token\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.685664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.685389    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-credential-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.685664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.685426    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-data\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.685664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.685451    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-credential-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.685664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.685477    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istiod-ca-cert\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.685664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.685499    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-podinfo\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.685664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.685561    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-token\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.685664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.685634    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-workload-certs\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.685664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.685653    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-workload-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.685664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.685668    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-data\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.686252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.685696    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7v6r\" (UniqueName: \"kubernetes.io/projected/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-kube-api-access-x7v6r\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.686252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.685724    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w8nh\" (UniqueName: \"kubernetes.io/projected/a4fd9180-f4aa-4986-b2f0-955296fe17ee-kube-api-access-2w8nh\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.686252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.685981    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istiod-ca-cert\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.687660 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.686288    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-workload-certs\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.687660 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.686511    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-workload-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.687660 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.686852    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-credential-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.687660 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.687454    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-data\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.688558 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.688533    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-envoy\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.689865 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.689843    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-podinfo\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.697828 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.697807    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-token\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.698041 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.698021    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w8nh\" (UniqueName: \"kubernetes.io/projected/a4fd9180-f4aa-4986-b2f0-955296fe17ee-kube-api-access-2w8nh\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.773855 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.773830    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:50.781981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.781955    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"]
Oct 30 19:22:50.786212 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:22:50.786182    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68953aed_0987_40c8_932c_fa4591f71831.slice/crio-d671fe75c25a1c03bf590563bac434388d009810e978f073e79ae25caa8325e6 WatchSource:0}: Error finding container d671fe75c25a1c03bf590563bac434388d009810e978f073e79ae25caa8325e6: Status 404 returned error can't find the container with id d671fe75c25a1c03bf590563bac434388d009810e978f073e79ae25caa8325e6
Oct 30 19:22:50.786679 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.786626    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-token\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.786784 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.786743    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-credential-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.786784 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.786777    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istiod-ca-cert\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.786965 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.786899    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-workload-certs\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.786965 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.786926    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-workload-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.786965 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.786949    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-data\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.787131 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.786975    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7v6r\" (UniqueName: \"kubernetes.io/projected/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-kube-api-access-x7v6r\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.787131 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.787028    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-envoy\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.787131 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.787078    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-podinfo\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.787345 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.787161    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-credential-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.787345 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.787251    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-workload-certs\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.787478 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.787454    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-data\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.787985 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.787961    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-workload-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.787985 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.787961    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istiod-ca-cert\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.789888 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.789867    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-podinfo\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.790082 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.790049    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:22:50.790146 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.790114    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:22:50.790215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.790147    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:22:50.790324 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.790303    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-envoy\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.798258 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.798240    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-token\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.799309 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.799289    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7v6r\" (UniqueName: \"kubernetes.io/projected/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-kube-api-access-x7v6r\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:50.872557 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:50.872344    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:51.097035 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:51.097008    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"]
Oct 30 19:22:51.121343 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:51.120522    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:22:51.121343 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:51.120603    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:22:51.121343 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:51.120649    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:22:51.253751 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:51.253594    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"]
Oct 30 19:22:51.256574 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:51.256508    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:22:51.256672 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:51.256593    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:22:51.256672 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:51.256636    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:22:51.682206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:51.675427    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n" event={"ID":"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d","Type":"ContainerStarted","Data":"81d6076e7f972a1be08e4feaf13fb91095394764249adff8a8909f31913def37"}
Oct 30 19:22:51.682206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:51.675565    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n" event={"ID":"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d","Type":"ContainerStarted","Data":"6d8cd183865d23df6af4510bd6664e1d0616dce0a848ac8472ad30d3cfd02527"}
Oct 30 19:22:51.682206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:51.678472    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp" event={"ID":"68953aed-0987-40c8-932c-fa4591f71831","Type":"ContainerStarted","Data":"c7445b8c4b6709aa85d73292565f09c31d7b4777baba0d05d4523c95baf2fce5"}
Oct 30 19:22:51.682206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:51.678509    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp" event={"ID":"68953aed-0987-40c8-932c-fa4591f71831","Type":"ContainerStarted","Data":"d671fe75c25a1c03bf590563bac434388d009810e978f073e79ae25caa8325e6"}
Oct 30 19:22:51.682206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:51.680618    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk" event={"ID":"a4fd9180-f4aa-4986-b2f0-955296fe17ee","Type":"ContainerStarted","Data":"da7324c6301810498df84c172f6661cc8934360a23252f40bc67b5bd2a8d9ce3"}
Oct 30 19:22:51.682206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:51.680667    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk" event={"ID":"a4fd9180-f4aa-4986-b2f0-955296fe17ee","Type":"ContainerStarted","Data":"d5fe525b875c63d46ff2d2b1030993046ccdad40c4a41bca3567ae31f92d71b2"}
Oct 30 19:22:51.745831 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:51.745733    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk" podStartSLOduration=1.745710698 podStartE2EDuration="1.745710698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:51.741154851 +0000 UTC m=+1003.482145932" watchObservedRunningTime="2026-10-30 19:22:51.745710698 +0000 UTC m=+1003.486701778"
Oct 30 19:22:51.749203 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:51.749115    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n" podStartSLOduration=1.749100331 podStartE2EDuration="1.749100331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:51.708465017 +0000 UTC m=+1003.449456096" watchObservedRunningTime="2026-10-30 19:22:51.749100331 +0000 UTC m=+1003.490091410"
Oct 30 19:22:51.805194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:51.791374    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:22:51.880191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:51.879576    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:22:52.461194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:52.460474    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:22:52.461194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:52.460595    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:22:52.461194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:52.460652    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp" podUID="68953aed-0987-40c8-932c-fa4591f71831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 19:22:52.777247 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:52.776880    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:22:52.777247 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:52.776938    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk" podUID="a4fd9180-f4aa-4986-b2f0-955296fe17ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 19:22:52.945194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:52.934522    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 19:22:52.945194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:52.934584    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n" podUID="555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 19:22:53.497209 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:53.461615    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:22:53.497209 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:53.461672    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp" podUID="68953aed-0987-40c8-932c-fa4591f71831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 19:22:53.789776 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:53.789679    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:22:53.790044 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:53.789994    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk" podUID="a4fd9180-f4aa-4986-b2f0-955296fe17ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 19:22:53.882711 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:53.882674    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 19:22:53.882968 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:53.882940    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n" podUID="555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 19:22:54.502200 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:54.501527    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:22:54.502200 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:54.501609    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp" podUID="68953aed-0987-40c8-932c-fa4591f71831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 19:22:54.782249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:54.777594    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:22:54.782249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:54.777668    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk" podUID="a4fd9180-f4aa-4986-b2f0-955296fe17ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 19:22:54.874611 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:54.874571    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 19:22:54.874900 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:54.874867    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n" podUID="555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 19:22:55.559758 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:55.528000    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:22:55.559758 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:55.528058    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp" podUID="68953aed-0987-40c8-932c-fa4591f71831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 19:22:55.778858 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:55.778822    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:22:55.779192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:55.779136    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk" podUID="a4fd9180-f4aa-4986-b2f0-955296fe17ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 19:22:55.875195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:55.874533    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 19:22:55.875195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:55.874594    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n" podUID="555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 19:22:56.459193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:56.458203    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:22:56.459193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:56.458274    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp" podUID="68953aed-0987-40c8-932c-fa4591f71831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 19:22:56.779242 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:56.778839    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:22:56.779701 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:56.779013    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk" podUID="a4fd9180-f4aa-4986-b2f0-955296fe17ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 19:22:56.884194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:56.883596    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 19:22:56.884194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:56.883659    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n" podUID="555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 19:22:57.457120 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:57.457082    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:22:57.457390 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:57.457360    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp" podUID="68953aed-0987-40c8-932c-fa4591f71831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 19:22:57.799247 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:57.789683    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:22:57.799247 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:57.789739    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk" podUID="a4fd9180-f4aa-4986-b2f0-955296fe17ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 19:22:57.900193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:57.896452    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 19:22:57.900193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:57.896510    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n" podUID="555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 19:22:58.457714 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:58.457671    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Oct 30 19:22:58.457938 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:58.457738    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp" podUID="68953aed-0987-40c8-932c-fa4591f71831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 19:22:58.782250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:58.775202    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:22:58.782250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:58.775360    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk" podUID="a4fd9180-f4aa-4986-b2f0-955296fe17ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": dial tcp 10.133.0.83:15021: connect: connection refused"
Oct 30 19:22:58.881251 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:58.873041    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused" start-of-body=
Oct 30 19:22:58.881251 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:22:58.873112    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n" podUID="555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": dial tcp 10.133.0.84:15021: connect: connection refused"
Oct 30 19:23:00.503194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:00.502680    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:23:00.503194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:00.502766    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp" podUID="68953aed-0987-40c8-932c-fa4591f71831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:23:00.692532 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:00.692498    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:23:00.733194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:00.708043    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:23:00.733584 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:00.733554    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:23:00.758539 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:00.756265    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp" podStartSLOduration=10.756243098 podStartE2EDuration="10.756243098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:51.773595693 +0000 UTC m=+1003.514586774" watchObservedRunningTime="2026-10-30 19:23:00.756243098 +0000 UTC m=+1012.497234178"
Oct 30 19:23:00.966963 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:00.966927    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:23:00.967266 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:00.967233    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n" podUID="555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:23:01.510301 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:01.510252    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:23:01.510910 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:01.510875    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp" podUID="68953aed-0987-40c8-932c-fa4591f71831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:23:01.532500 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:01.532469    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:23:01.538462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:01.538438    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:23:01.560860 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:01.560830    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:23:02.568291 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:02.568253    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:23:02.568903 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:02.568869    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp" podUID="68953aed-0987-40c8-932c-fa4591f71831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:23:03.205953 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:03.205914    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:23:03.207543 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:03.207519    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:23:03.209997 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:03.209164    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:23:04.851266 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:04.851224    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"]
Oct 30 19:23:04.866192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:04.865530    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:04.866192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:04.866185    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"]
Oct 30 19:23:04.943315 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:04.943283    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-workload-socket\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:04.943491 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:04.943338    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83f517e5-9b8e-425a-845b-56253704cd78-istio-token\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:04.943556 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:04.943518    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-workload-certs\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:04.943613 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:04.943582    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83f517e5-9b8e-425a-845b-56253704cd78-istio-podinfo\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:04.943672 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:04.943646    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-istio-data\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:04.943758 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:04.943743    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-istio-envoy\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:04.943833 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:04.943775    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtdbz\" (UniqueName: \"kubernetes.io/projected/83f517e5-9b8e-425a-845b-56253704cd78-kube-api-access-gtdbz\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:04.943833 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:04.943809    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83f517e5-9b8e-425a-845b-56253704cd78-istiod-ca-cert\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:04.943900 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:04.943875    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-credential-socket\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:05.044964 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.044933    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-workload-certs\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:05.045137 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.044991    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83f517e5-9b8e-425a-845b-56253704cd78-istio-podinfo\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:05.045137 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.045045    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-istio-data\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:05.045137 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.045110    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-istio-envoy\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:05.045137 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.045135    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtdbz\" (UniqueName: \"kubernetes.io/projected/83f517e5-9b8e-425a-845b-56253704cd78-kube-api-access-gtdbz\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:05.045411 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.045162    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83f517e5-9b8e-425a-845b-56253704cd78-istiod-ca-cert\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:05.045411 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.045276    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-credential-socket\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:05.045411 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.045341    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-workload-socket\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:05.045411 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.045383    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83f517e5-9b8e-425a-845b-56253704cd78-istio-token\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:05.045611 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.045458    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-istio-data\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:05.045737 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.045713    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-credential-socket\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:05.045827 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.045808    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-workload-socket\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:05.045952 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.045934    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83f517e5-9b8e-425a-845b-56253704cd78-istiod-ca-cert\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:05.046022 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.046010    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-workload-certs\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:05.047902 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.047877    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-istio-envoy\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:05.048243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.048222    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83f517e5-9b8e-425a-845b-56253704cd78-istio-podinfo\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:05.058572 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.058543    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83f517e5-9b8e-425a-845b-56253704cd78-istio-token\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:05.058710 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.058690    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtdbz\" (UniqueName: \"kubernetes.io/projected/83f517e5-9b8e-425a-845b-56253704cd78-kube-api-access-gtdbz\") pod \"e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") " pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:05.185205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.185114    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:05.289822 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.289786    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"]
Oct 30 19:23:05.299208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.298387    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.316385 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.316340    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"]
Oct 30 19:23:05.327275 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.327244    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"]
Oct 30 19:23:05.332761 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.332737    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"]
Oct 30 19:23:05.332894 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.332833    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.348763 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.348740    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-credential-socket\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.348917 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.348793    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-workload-socket\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.348917 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.348845    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-data\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.348917 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.348870    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istiod-ca-cert\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.349144 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.348934    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-token\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.349144 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.348970    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-podinfo\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.349144 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.349026    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cd4t\" (UniqueName: \"kubernetes.io/projected/1d63b36f-3cfa-4438-8fb1-21480c9e9527-kube-api-access-7cd4t\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.349144 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.349102    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-workload-certs\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.349144 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.349129    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-envoy\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.450090 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.450000    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-workload-certs\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.450090 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.450043    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-envoy\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.450090 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.450065    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-data\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.450411 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.450116    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-workload-certs\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.453408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.450235    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-envoy\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.453408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.450836    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-token\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.453408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.450899    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-credential-socket\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.453408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.451256    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-credential-socket\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.453408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.451418    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-workload-socket\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.453408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.451497    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-workload-socket\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.453408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.451536    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-data\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.453408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.451567    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istiod-ca-cert\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.453408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.451598    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q8t8\" (UniqueName: \"kubernetes.io/projected/3b269ee8-7299-4e79-8f77-ed5e4e76730d-kube-api-access-5q8t8\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.453408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.451719    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-token\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.453408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.451750    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istiod-ca-cert\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.453408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.451798    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-podinfo\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.453408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.451836    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-credential-socket\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.453408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.451927    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cd4t\" (UniqueName: \"kubernetes.io/projected/1d63b36f-3cfa-4438-8fb1-21480c9e9527-kube-api-access-7cd4t\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.453408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.451999    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-podinfo\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.453408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.452446    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-workload-socket\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.453408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.453050    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-data\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.454839 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.454235    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-envoy\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.454839 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.454610    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istiod-ca-cert\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.462443 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.462416    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-workload-certs\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.481039 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.472599    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-podinfo\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.481039 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.475131    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cd4t\" (UniqueName: \"kubernetes.io/projected/1d63b36f-3cfa-4438-8fb1-21480c9e9527-kube-api-access-7cd4t\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.481039 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.476629    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-token\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.495415 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:23:05.495326    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83f517e5_9b8e_425a_845b_56253704cd78.slice/crio-e444c31ab6a997cc550eeb90aba60f33205fbb8928e41438f9d6023e1d49ebf2 WatchSource:0}: Error finding container e444c31ab6a997cc550eeb90aba60f33205fbb8928e41438f9d6023e1d49ebf2: Status 404 returned error can't find the container with id e444c31ab6a997cc550eeb90aba60f33205fbb8928e41438f9d6023e1d49ebf2
Oct 30 19:23:05.496569 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.496544    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"]
Oct 30 19:23:05.500097 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.500053    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:23:05.500229 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.500124    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:23:05.500229 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.500154    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:23:05.569685 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.564663    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-podinfo\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.569685 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.564747    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-data\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.569685 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.564829    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-workload-certs\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.569685 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.564865    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-envoy\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.569685 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.564950    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-token\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.569685 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.565039    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-workload-socket\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.569685 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.565071    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q8t8\" (UniqueName: \"kubernetes.io/projected/3b269ee8-7299-4e79-8f77-ed5e4e76730d-kube-api-access-5q8t8\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.569685 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.565149    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istiod-ca-cert\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.569685 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.565206    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-credential-socket\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.569685 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.565772    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-credential-socket\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.584321 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.578932    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-data\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.584321 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.579260    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-workload-certs\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.584321 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.580108    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-workload-socket\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.584321 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.580732    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istiod-ca-cert\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.602875 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.602841    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q8t8\" (UniqueName: \"kubernetes.io/projected/3b269ee8-7299-4e79-8f77-ed5e4e76730d-kube-api-access-5q8t8\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.603831 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.603092    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-envoy\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.603831 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.603487    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-podinfo\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.628940 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.628878    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-token\") pod \"e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") " pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.641223 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.640855    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:05.657499 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.657249    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:05.953404 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.953366    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"]
Oct 30 19:23:05.954374 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:23:05.954346    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d63b36f_3cfa_4438_8fb1_21480c9e9527.slice/crio-e3140b4172c423843d76347bc99b5a5ef10f610d3449aee6c0f3481f00eedb67 WatchSource:0}: Error finding container e3140b4172c423843d76347bc99b5a5ef10f610d3449aee6c0f3481f00eedb67: Status 404 returned error can't find the container with id e3140b4172c423843d76347bc99b5a5ef10f610d3449aee6c0f3481f00eedb67
Oct 30 19:23:05.957674 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.957644    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:23:05.957747 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.957711    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:23:05.957747 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.957741    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:23:05.983267 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.982862    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"]
Oct 30 19:23:05.985353 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:23:05.985321    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b269ee8_7299_4e79_8f77_ed5e4e76730d.slice/crio-f3b3eeb14592ccbb469605c9ceed609858eae0537993b658e5c9bd509742dc97 WatchSource:0}: Error finding container f3b3eeb14592ccbb469605c9ceed609858eae0537993b658e5c9bd509742dc97: Status 404 returned error can't find the container with id f3b3eeb14592ccbb469605c9ceed609858eae0537993b658e5c9bd509742dc97
Oct 30 19:23:05.988770 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.988738    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:23:05.988991 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.988819    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:23:05.988991 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:05.988869    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:23:06.443856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:06.443820    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9" event={"ID":"83f517e5-9b8e-425a-845b-56253704cd78","Type":"ContainerStarted","Data":"b71879018815d86e282d8f0e69001291a1091babd9d025219863a4f39a8df76d"}
Oct 30 19:23:06.444030 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:06.443873    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9" event={"ID":"83f517e5-9b8e-425a-845b-56253704cd78","Type":"ContainerStarted","Data":"e444c31ab6a997cc550eeb90aba60f33205fbb8928e41438f9d6023e1d49ebf2"}
Oct 30 19:23:06.446417 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:06.446392    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6" event={"ID":"1d63b36f-3cfa-4438-8fb1-21480c9e9527","Type":"ContainerStarted","Data":"4ab11722d99de13c19ffed2c9c210735bdc599ea12cd356ac3ccb191ab7dadfb"}
Oct 30 19:23:06.446469 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:06.446455    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6" event={"ID":"1d63b36f-3cfa-4438-8fb1-21480c9e9527","Type":"ContainerStarted","Data":"e3140b4172c423843d76347bc99b5a5ef10f610d3449aee6c0f3481f00eedb67"}
Oct 30 19:23:06.448535 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:06.448511    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4" event={"ID":"3b269ee8-7299-4e79-8f77-ed5e4e76730d","Type":"ContainerStarted","Data":"dc37e825ba702110e87ec82948eddc602349e7d442cc1dd540e1a126046d1d6f"}
Oct 30 19:23:06.448666 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:06.448548    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4" event={"ID":"3b269ee8-7299-4e79-8f77-ed5e4e76730d","Type":"ContainerStarted","Data":"f3b3eeb14592ccbb469605c9ceed609858eae0537993b658e5c9bd509742dc97"}
Oct 30 19:23:06.486216 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:06.477740    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9" podStartSLOduration=2.477722077 podStartE2EDuration="2.477722077s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:23:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:23:06.465125461 +0000 UTC m=+1018.206116540" watchObservedRunningTime="2026-10-30 19:23:06.477722077 +0000 UTC m=+1018.218713158"
Oct 30 19:23:06.550727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:06.550649    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4" podStartSLOduration=1.550630681 podStartE2EDuration="1.550630681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:23:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:23:06.526561732 +0000 UTC m=+1018.267552811" watchObservedRunningTime="2026-10-30 19:23:06.550630681 +0000 UTC m=+1018.291621760"
Oct 30 19:23:06.554460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:06.554411    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6" podStartSLOduration=1.554398374 podStartE2EDuration="1.554398374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:23:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:23:06.548516682 +0000 UTC m=+1018.289507791" watchObservedRunningTime="2026-10-30 19:23:06.554398374 +0000 UTC m=+1018.295389453"
Oct 30 19:23:06.641405 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:06.641366    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:06.659323 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:06.659291    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:07.198388 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:07.197919    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:07.198388 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:07.198044    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:23:07.198388 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:07.198092    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9" podUID="83f517e5-9b8e-425a-845b-56253704cd78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 19:23:07.650237 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:07.647024    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 19:23:07.650237 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:07.647106    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6" podUID="1d63b36f-3cfa-4438-8fb1-21480c9e9527" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 19:23:07.661298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:07.661263    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:23:07.661582 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:07.661550    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4" podUID="3b269ee8-7299-4e79-8f77-ed5e4e76730d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 19:23:08.187579 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:08.187542    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:23:08.187876 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:08.187844    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9" podUID="83f517e5-9b8e-425a-845b-56253704cd78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 19:23:08.715227 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:08.713652    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 19:23:08.715227 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:08.713716    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6" podUID="1d63b36f-3cfa-4438-8fb1-21480c9e9527" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 19:23:08.715227 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:08.714562    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:23:08.715227 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:08.714599    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4" podUID="3b269ee8-7299-4e79-8f77-ed5e4e76730d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 19:23:09.215193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:09.209415    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:23:09.215193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:09.209476    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9" podUID="83f517e5-9b8e-425a-845b-56253704cd78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 19:23:09.646064 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:09.644610    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 19:23:09.646064 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:09.644672    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6" podUID="1d63b36f-3cfa-4438-8fb1-21480c9e9527" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 19:23:09.673203 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:09.672687    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:23:09.673203 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:09.672753    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4" podUID="3b269ee8-7299-4e79-8f77-ed5e4e76730d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 19:23:10.188194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:10.185559    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:23:10.188194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:10.185623    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9" podUID="83f517e5-9b8e-425a-845b-56253704cd78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 19:23:10.649624 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:10.649585    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 19:23:10.649893 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:10.649862    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6" podUID="1d63b36f-3cfa-4438-8fb1-21480c9e9527" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 19:23:10.659046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:10.659018    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:23:10.659327 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:10.659301    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4" podUID="3b269ee8-7299-4e79-8f77-ed5e4e76730d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 19:23:11.195976 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:11.195936    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:23:11.209311 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:11.209262    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9" podUID="83f517e5-9b8e-425a-845b-56253704cd78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 19:23:11.646195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:11.646142    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 19:23:11.646372 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:11.646215    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6" podUID="1d63b36f-3cfa-4438-8fb1-21480c9e9527" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 19:23:11.664194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:11.660978    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:23:11.664194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:11.661043    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4" podUID="3b269ee8-7299-4e79-8f77-ed5e4e76730d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 19:23:12.186194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:12.185541    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:23:12.186194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:12.185603    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9" podUID="83f517e5-9b8e-425a-845b-56253704cd78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 19:23:12.643194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:12.642385    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 19:23:12.643194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:12.642458    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6" podUID="1d63b36f-3cfa-4438-8fb1-21480c9e9527" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 19:23:12.658193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:12.658133    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:23:12.658437 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:12.658405    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4" podUID="3b269ee8-7299-4e79-8f77-ed5e4e76730d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 19:23:13.186422 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:13.186377    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 19:23:13.186641 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:13.186449    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9" podUID="83f517e5-9b8e-425a-845b-56253704cd78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 19:23:13.641924 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:13.641886    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 19:23:13.642222 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:13.642186    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6" podUID="1d63b36f-3cfa-4438-8fb1-21480c9e9527" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 19:23:13.658466 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:13.658440    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 19:23:13.659155 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:13.659127    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4" podUID="3b269ee8-7299-4e79-8f77-ed5e4e76730d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 19:23:14.375383 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:14.375351    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:14.390290 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:14.390252    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:14.476505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:14.475641    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:23:14.890475 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:14.890428    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:14.891451 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:14.891429    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:14.892377 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:14.892358    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:23:15.035972 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:15.035941    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:15.037835 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:15.037774    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:15.038441 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:15.038420    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:23:15.667845 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:23:15.667811    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:23:20.207327 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.207287    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"]
Oct 30 19:23:20.213184 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.213147    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.228159 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.228126    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"]
Oct 30 19:23:20.264872 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.264789    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-podinfo\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.264872 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.264844    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-workload-socket\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.265084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.264947    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-workload-certs\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.265084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.265025    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-data\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.265192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.265153    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-envoy\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.265317 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.265289    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-credential-socket\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.265401 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.265379    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-token\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.265448 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.265432    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj7b8\" (UniqueName: \"kubernetes.io/projected/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-kube-api-access-dj7b8\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.265512 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.265491    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istiod-ca-cert\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.372202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.369275    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-data\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.372202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.369370    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-envoy\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.372202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.369452    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-credential-socket\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.372202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.369488    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-token\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.372202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.369535    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj7b8\" (UniqueName: \"kubernetes.io/projected/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-kube-api-access-dj7b8\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.372202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.369588    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istiod-ca-cert\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.372202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.369762    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-podinfo\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.372202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.369796    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-workload-socket\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.372202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.369825    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-workload-certs\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.372202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.370689    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-workload-certs\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.372202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.371162    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-data\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.394204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.383578    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-credential-socket\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.394204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.384924    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-workload-socket\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.394204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.385685    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istiod-ca-cert\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.419695 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.419312    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-token\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.421871 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.421818    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-podinfo\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.424903 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.422373    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-envoy\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.428706 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.428680    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj7b8\" (UniqueName: \"kubernetes.io/projected/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-kube-api-access-dj7b8\") pod \"e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") " pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.531858 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.531769    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:20.836233 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.836206    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"]
Oct 30 19:23:20.837075 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:23:20.837047    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e1416a9_3d41_4166_bc0a_01dbc28aad2d.slice/crio-699474c6f5578ff9d96438bd66a54ae59b6560f63c3551516dd48574995ba036 WatchSource:0}: Error finding container 699474c6f5578ff9d96438bd66a54ae59b6560f63c3551516dd48574995ba036: Status 404 returned error can't find the container with id 699474c6f5578ff9d96438bd66a54ae59b6560f63c3551516dd48574995ba036
Oct 30 19:23:20.839631 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.839602    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:23:20.839720 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.839668    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:23:20.839720 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:20.839703    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:23:21.832422 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:21.832382    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g" event={"ID":"0e1416a9-3d41-4166-bc0a-01dbc28aad2d","Type":"ContainerStarted","Data":"7e6640edd3830652832478998218c506a60327e2b82e3d167e54510cb88e37d5"}
Oct 30 19:23:21.832829 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:21.832461    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g" event={"ID":"0e1416a9-3d41-4166-bc0a-01dbc28aad2d","Type":"ContainerStarted","Data":"699474c6f5578ff9d96438bd66a54ae59b6560f63c3551516dd48574995ba036"}
Oct 30 19:23:21.863667 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:21.863620    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g" podStartSLOduration=1.863606902 podStartE2EDuration="1.863606902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:23:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:23:21.859456205 +0000 UTC m=+1033.600447285" watchObservedRunningTime="2026-10-30 19:23:21.863606902 +0000 UTC m=+1033.604597981"
Oct 30 19:23:22.532053 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:22.532021    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:22.533387 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:22.533360    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:23:22.533521 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:22.533419    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g" podUID="0e1416a9-3d41-4166-bc0a-01dbc28aad2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 19:23:23.532592 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:23.532541    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:23:23.533092 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:23.532635    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g" podUID="0e1416a9-3d41-4166-bc0a-01dbc28aad2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 19:23:24.536099 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:24.536062    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:23:24.536709 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:24.536656    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g" podUID="0e1416a9-3d41-4166-bc0a-01dbc28aad2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 19:23:25.538195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:25.537240    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:23:25.538195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:25.537313    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g" podUID="0e1416a9-3d41-4166-bc0a-01dbc28aad2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 19:23:26.687219 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:26.675897    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:23:26.687219 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:26.675958    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g" podUID="0e1416a9-3d41-4166-bc0a-01dbc28aad2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 19:23:27.548194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:27.536732    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:23:27.548194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:27.536806    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g" podUID="0e1416a9-3d41-4166-bc0a-01dbc28aad2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 19:23:28.463879 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:28.463840    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:23:28.464492 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:28.464459    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp" podUID="e6dd71b4-1741-4da3-99f4-82121e585d89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:23:28.542393 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:28.535214    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:23:28.542393 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:28.535283    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g" podUID="0e1416a9-3d41-4166-bc0a-01dbc28aad2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 19:23:29.593849 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:29.593811    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:23:29.594478 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:29.594442    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g" podUID="0e1416a9-3d41-4166-bc0a-01dbc28aad2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 19:23:29.642746 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:29.642711    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:23:29.643031 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:29.643001    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd" podUID="e53dd061-8aa9-4ace-9140-aea8a7375c20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:23:30.475217 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:30.458323    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:23:30.476020 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:30.475536    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9" podUID="83f517e5-9b8e-425a-845b-56253704cd78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:23:30.566639 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:30.566606    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:23:30.628920 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:30.628870    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g" podUID="0e1416a9-3d41-4166-bc0a-01dbc28aad2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 19:23:31.532672 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:31.532638    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:23:31.532844 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:31.532692    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g" podUID="0e1416a9-3d41-4166-bc0a-01dbc28aad2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 19:23:32.532645 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:32.532603    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 19:23:32.546112 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:32.533250    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g" podUID="0e1416a9-3d41-4166-bc0a-01dbc28aad2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 19:23:34.576745 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:34.576296    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:23:34.576745 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:34.576348    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g" podUID="0e1416a9-3d41-4166-bc0a-01dbc28aad2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:23:35.009573 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:35.009535    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:35.011085 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:35.011056    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:35.025677 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:35.025647    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:23:46.010800 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:23:46.009420    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:23:48.758196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:48.755122    4524 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz" podUID="e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:23:49.398461 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:49.398425    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:23:49.398782 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:49.398752    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp" podUID="68953aed-0987-40c8-932c-fa4591f71831" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:23:49.891333 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:49.891292    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:23:49.891942 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:49.891910    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb" podUID="849295c8-38ec-4630-95be-60a6200e4a20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:23:50.927191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:50.921957    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"]
Oct 30 19:23:51.300005 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.299927    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.300414 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.300313    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"]
Oct 30 19:23:51.389534 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.389493    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bb42l\" (UniqueName: \"kubernetes.io/projected/d72b215c-1d3b-400c-9238-906b0199f4c7-kube-api-access-bb42l\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.389815 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.389791    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-envoy\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.390008 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.389990    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-podinfo\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.390205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.390186    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-workload-certs\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.390424 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.390403    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-workload-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.390573 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.390557    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-credential-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.390686 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.390671    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d72b215c-1d3b-400c-9238-906b0199f4c7-istiod-ca-cert\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.390811 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.390795    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-data\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.390957 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.390942    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-token\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.492615 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.492577    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-workload-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.492804 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.492648    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-credential-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.492804 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.492665    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d72b215c-1d3b-400c-9238-906b0199f4c7-istiod-ca-cert\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.492997 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.492965    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-data\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.493148 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.493045    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-workload-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.493148 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.493106    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-token\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.493502 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.493470    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-credential-socket\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.494202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.494158    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d72b215c-1d3b-400c-9238-906b0199f4c7-istiod-ca-cert\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.494316 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.494233    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bb42l\" (UniqueName: \"kubernetes.io/projected/d72b215c-1d3b-400c-9238-906b0199f4c7-kube-api-access-bb42l\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.494316 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.494297    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-envoy\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.494463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.494447    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-podinfo\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.494634 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.494615    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-workload-certs\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.494879 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.494859    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-data\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.495393 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.495361    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-workload-certs\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.496858 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.496831    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-envoy\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.498065 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.498039    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-podinfo\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.510131 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.510105    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bb42l\" (UniqueName: \"kubernetes.io/projected/d72b215c-1d3b-400c-9238-906b0199f4c7-kube-api-access-bb42l\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.523104 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.522888    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-token\") pod \"e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") " pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:51.663199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:51.658410    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:52.022541 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:52.022509    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"]
Oct 30 19:23:52.023265 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:23:52.023242    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd72b215c_1d3b_400c_9238_906b0199f4c7.slice/crio-67617be13bda60b93f3c01a7cf054aeb43854f4efc85f14253b934047394e501 WatchSource:0}: Error finding container 67617be13bda60b93f3c01a7cf054aeb43854f4efc85f14253b934047394e501: Status 404 returned error can't find the container with id 67617be13bda60b93f3c01a7cf054aeb43854f4efc85f14253b934047394e501
Oct 30 19:23:52.025781 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:52.025733    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:23:52.025879 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:52.025811    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:23:52.025879 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:52.025858    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:23:52.843082 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:52.843045    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc" event={"ID":"d72b215c-1d3b-400c-9238-906b0199f4c7","Type":"ContainerStarted","Data":"47ce3a2c33bc18d4f462cdf8af896c1076f1b432f8f2d4e42f5d27d545f8a72d"}
Oct 30 19:23:52.843302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:52.843097    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc" event={"ID":"d72b215c-1d3b-400c-9238-906b0199f4c7","Type":"ContainerStarted","Data":"67617be13bda60b93f3c01a7cf054aeb43854f4efc85f14253b934047394e501"}
Oct 30 19:23:52.874191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:52.867554    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc" podStartSLOduration=2.867537081 podStartE2EDuration="2.867537081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:23:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:23:52.865151218 +0000 UTC m=+1064.606142300" watchObservedRunningTime="2026-10-30 19:23:52.867537081 +0000 UTC m=+1064.608528162"
Oct 30 19:23:53.659028 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:53.658995    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:53.661030 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:53.661005    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 19:23:53.661151 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:53.661052    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc" podUID="d72b215c-1d3b-400c-9238-906b0199f4c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 19:23:54.659774 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:54.659738    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 19:23:54.660192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:54.659811    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc" podUID="d72b215c-1d3b-400c-9238-906b0199f4c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 19:23:55.664191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:55.662050    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 19:23:55.664191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:55.662125    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc" podUID="d72b215c-1d3b-400c-9238-906b0199f4c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 19:23:56.660621 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:56.660587    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 19:23:56.660801 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:56.660641    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc" podUID="d72b215c-1d3b-400c-9238-906b0199f4c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 19:23:57.659894 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:57.659857    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 19:23:57.660386 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:57.659915    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc" podUID="d72b215c-1d3b-400c-9238-906b0199f4c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 19:23:58.659843 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:58.659810    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused" start-of-body=
Oct 30 19:23:58.660012 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:58.659864    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc" podUID="d72b215c-1d3b-400c-9238-906b0199f4c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": dial tcp 10.133.0.89:15021: connect: connection refused"
Oct 30 19:23:59.774742 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:59.774700    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:59.776216 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:59.776194    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:23:59.776945 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:23:59.776925    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:24:04.926721 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:04.926678    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:24:04.946315 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:04.946273    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb" podUID="849295c8-38ec-4630-95be-60a6200e4a20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:24:06.076369 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:06.076316    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:24:06.084268 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:06.084209    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g" podUID="0e1416a9-3d41-4166-bc0a-01dbc28aad2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:24:14.777884 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:14.777845    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8867-fe7e70"]
Oct 30 19:24:14.782915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:14.782890    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8867-fe7e70"
Oct 30 19:24:14.790107 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:14.790076    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8867-fe7e70"]
Oct 30 19:24:14.790663 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:14.790641    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-bmmlc\""
Oct 30 19:24:14.823536 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:14.823504    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cx6q\" (UniqueName: \"kubernetes.io/projected/f130e864-1132-4281-a8a0-ca2b323bb22e-kube-api-access-8cx6q\") pod \"test-curl-8867-fe7e70\" (UID: \"f130e864-1132-4281-a8a0-ca2b323bb22e\") " pod="opendatahub/test-curl-8867-fe7e70"
Oct 30 19:24:14.924674 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:14.924639    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cx6q\" (UniqueName: \"kubernetes.io/projected/f130e864-1132-4281-a8a0-ca2b323bb22e-kube-api-access-8cx6q\") pod \"test-curl-8867-fe7e70\" (UID: \"f130e864-1132-4281-a8a0-ca2b323bb22e\") " pod="opendatahub/test-curl-8867-fe7e70"
Oct 30 19:24:14.936520 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:14.936494    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cx6q\" (UniqueName: \"kubernetes.io/projected/f130e864-1132-4281-a8a0-ca2b323bb22e-kube-api-access-8cx6q\") pod \"test-curl-8867-fe7e70\" (UID: \"f130e864-1132-4281-a8a0-ca2b323bb22e\") " pod="opendatahub/test-curl-8867-fe7e70"
Oct 30 19:24:15.102537 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:15.102505    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8867-fe7e70"
Oct 30 19:24:15.507155 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:15.507125    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8867-fe7e70"]
Oct 30 19:24:15.507613 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:24:15.507586    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf130e864_1132_4281_a8a0_ca2b323bb22e.slice/crio-2b6f7ab4ba4db55edf54653c418b8be742340546429c541335d65b8146fae3e3 WatchSource:0}: Error finding container 2b6f7ab4ba4db55edf54653c418b8be742340546429c541335d65b8146fae3e3: Status 404 returned error can't find the container with id 2b6f7ab4ba4db55edf54653c418b8be742340546429c541335d65b8146fae3e3
Oct 30 19:24:15.510024 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:15.510004    4524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:24:15.628388 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:24:15.628325    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:24:15.945873 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:15.945833    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8867-fe7e70" event={"ID":"f130e864-1132-4281-a8a0-ca2b323bb22e","Type":"ContainerStarted","Data":"2b6f7ab4ba4db55edf54653c418b8be742340546429c541335d65b8146fae3e3"}
Oct 30 19:24:16.954666 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:16.954634    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8867-fe7e70" event={"ID":"f130e864-1132-4281-a8a0-ca2b323bb22e","Type":"ContainerStarted","Data":"ba1173c118cb5d0e87da1577026f861e851928a088c889492fae2f7ee405061f"}
Oct 30 19:24:16.974080 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:16.974020    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-8867-fe7e70" podStartSLOduration=2.195118897 podStartE2EDuration="2.974000011s" totalImagesPullingTime="778.881114ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:24:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:24:16.96930435 +0000 UTC m=+1088.710295429" watchObservedRunningTime="2026-10-30 19:24:16.974000011 +0000 UTC m=+1088.714991089"
Oct 30 19:24:17.767918 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:17.767840    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8867-fe7e70"]
Oct 30 19:24:17.774785 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:17.774729    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8867-fe7e70"]
Oct 30 19:24:17.932608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:17.932575    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8867-43cc7d"]
Oct 30 19:24:17.933268 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:17.933254    4524 cpu_manager.go:414] "containerMap: removing container" podUID="f130e864-1132-4281-a8a0-ca2b323bb22e" containerName="test-curl-8867-fe7e70"
Oct 30 19:24:17.933268 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:17.933269    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f130e864-1132-4281-a8a0-ca2b323bb22e" containerName="test-curl-8867-fe7e70"
Oct 30 19:24:17.933394 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:17.933385    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="f130e864-1132-4281-a8a0-ca2b323bb22e" containerName="test-curl-8867-fe7e70"
Oct 30 19:24:17.937319 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:17.937298    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8867-43cc7d"
Oct 30 19:24:17.944712 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:17.944677    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8867-43cc7d"]
Oct 30 19:24:18.028484 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:18.028384    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wpjj\" (UniqueName: \"kubernetes.io/projected/7d3fbd80-1418-4464-943d-ea5a77165dc9-kube-api-access-8wpjj\") pod \"test-curl-8867-43cc7d\" (UID: \"7d3fbd80-1418-4464-943d-ea5a77165dc9\") " pod="opendatahub/test-curl-8867-43cc7d"
Oct 30 19:24:18.129479 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:18.129443    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wpjj\" (UniqueName: \"kubernetes.io/projected/7d3fbd80-1418-4464-943d-ea5a77165dc9-kube-api-access-8wpjj\") pod \"test-curl-8867-43cc7d\" (UID: \"7d3fbd80-1418-4464-943d-ea5a77165dc9\") " pod="opendatahub/test-curl-8867-43cc7d"
Oct 30 19:24:18.148731 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:18.148699    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wpjj\" (UniqueName: \"kubernetes.io/projected/7d3fbd80-1418-4464-943d-ea5a77165dc9-kube-api-access-8wpjj\") pod \"test-curl-8867-43cc7d\" (UID: \"7d3fbd80-1418-4464-943d-ea5a77165dc9\") " pod="opendatahub/test-curl-8867-43cc7d"
Oct 30 19:24:18.254705 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:18.254670    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8867-43cc7d"
Oct 30 19:24:18.635632 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:18.635601    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8867-43cc7d"]
Oct 30 19:24:18.994224 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:18.994164    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-8867-fe7e70" podUID="f130e864-1132-4281-a8a0-ca2b323bb22e" containerName="test-curl-8867-fe7e70" containerID="cri-o://ba1173c118cb5d0e87da1577026f861e851928a088c889492fae2f7ee405061f" gracePeriod=2
Oct 30 19:24:18.995006 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:18.994977    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8867-43cc7d" event={"ID":"7d3fbd80-1418-4464-943d-ea5a77165dc9","Type":"ContainerStarted","Data":"9663c224f64ab42c32515bd5ccfc8a82933901e793e1a24dabb66f28f6adeb2b"}
Oct 30 19:24:18.995245 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:18.995229    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8867-43cc7d" event={"ID":"7d3fbd80-1418-4464-943d-ea5a77165dc9","Type":"ContainerStarted","Data":"59824d547a80e53bd93d76ea61366ed7372abbf84d303563754b5a5c6d7d7a19"}
Oct 30 19:24:19.929516 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:19.927544    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-8867-43cc7d" podStartSLOduration=2.927522801 podStartE2EDuration="2.927522801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:24:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:24:19.022305256 +0000 UTC m=+1090.763296336" watchObservedRunningTime="2026-10-30 19:24:19.927522801 +0000 UTC m=+1091.668513879"
Oct 30 19:24:19.991193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:19.990311    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8867-43cc7d"]
Oct 30 19:24:20.059796 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:20.057149    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8867-43cc7d"]
Oct 30 19:24:21.057007 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:21.056982    4524 generic.go:356] "Generic (PLEG): container finished" podID="f130e864-1132-4281-a8a0-ca2b323bb22e" containerID="ba1173c118cb5d0e87da1577026f861e851928a088c889492fae2f7ee405061f" exitCode=137
Oct 30 19:24:21.057374 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:21.057129    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-8867-43cc7d" podUID="7d3fbd80-1418-4464-943d-ea5a77165dc9" containerName="test-curl-8867-43cc7d" containerID="cri-o://9663c224f64ab42c32515bd5ccfc8a82933901e793e1a24dabb66f28f6adeb2b" gracePeriod=2
Oct 30 19:24:21.234651 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:21.234624    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8867-fe7e70"
Oct 30 19:24:21.305915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:21.305835    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f130e864-1132-4281-a8a0-ca2b323bb22e-kube-api-access-8cx6q\" (UniqueName: \"kubernetes.io/projected/f130e864-1132-4281-a8a0-ca2b323bb22e-kube-api-access-8cx6q\") pod \"f130e864-1132-4281-a8a0-ca2b323bb22e\" (UID: \"f130e864-1132-4281-a8a0-ca2b323bb22e\") "
Oct 30 19:24:21.308010 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:21.307975    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f130e864-1132-4281-a8a0-ca2b323bb22e-kube-api-access-8cx6q" pod "f130e864-1132-4281-a8a0-ca2b323bb22e" (UID: "f130e864-1132-4281-a8a0-ca2b323bb22e"). InnerVolumeSpecName "kube-api-access-8cx6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:21.407517 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:21.407486    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cx6q\" (UniqueName: \"kubernetes.io/projected/f130e864-1132-4281-a8a0-ca2b323bb22e-kube-api-access-8cx6q\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:22.250198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.241986    4524 scope.go:122] "RemoveContainer" containerID="ba1173c118cb5d0e87da1577026f861e851928a088c889492fae2f7ee405061f"
Oct 30 19:24:22.250198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.242259    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8867-fe7e70"
Oct 30 19:24:22.402891 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.402854    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f130e864-1132-4281-a8a0-ca2b323bb22e" path="/var/lib/kubelet/pods/f130e864-1132-4281-a8a0-ca2b323bb22e/volumes"
Oct 30 19:24:22.404187 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:24:22.404143    4524 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-8867-fe7e70\" is forbidden: User \"system:node:ip-10-0-137-200.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-200.us-east-2.compute.internal' and this object" podUID="f130e864-1132-4281-a8a0-ca2b323bb22e" pod="opendatahub/test-curl-8867-fe7e70"
Oct 30 19:24:22.616538 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.616501    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl"]
Oct 30 19:24:22.617151 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.617138    4524 cpu_manager.go:414] "containerMap: removing container" podUID="7d3fbd80-1418-4464-943d-ea5a77165dc9" containerName="test-curl-8867-43cc7d"
Oct 30 19:24:22.617242 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.617153    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d3fbd80-1418-4464-943d-ea5a77165dc9" containerName="test-curl-8867-43cc7d"
Oct 30 19:24:22.617297 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.617287    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d3fbd80-1418-4464-943d-ea5a77165dc9" containerName="test-curl-8867-43cc7d"
Oct 30 19:24:22.622519 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.622498    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl"
Oct 30 19:24:22.630536 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.630512    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-250fb057\""
Oct 30 19:24:22.632598 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.632571    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl"]
Oct 30 19:24:22.690159 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.690126    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp"]
Oct 30 19:24:22.694440 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.694420    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp"
Oct 30 19:24:22.700975 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.700940    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-250fb057-dockercfg-d9rd8\""
Oct 30 19:24:22.703001 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.702975    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f6a5962-7e97-4b7d-af82-aa05f7d0785f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl\" (UID: \"4f6a5962-7e97-4b7d-af82-aa05f7d0785f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl"
Oct 30 19:24:22.703259 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.703221    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gldn7\" (UniqueName: \"kubernetes.io/projected/4f6a5962-7e97-4b7d-af82-aa05f7d0785f-kube-api-access-gldn7\") pod \"payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl\" (UID: \"4f6a5962-7e97-4b7d-af82-aa05f7d0785f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl"
Oct 30 19:24:22.706205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.706183    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp"]
Oct 30 19:24:22.851015 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.843073    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f6a5962-7e97-4b7d-af82-aa05f7d0785f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl\" (UID: \"4f6a5962-7e97-4b7d-af82-aa05f7d0785f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl"
Oct 30 19:24:22.851015 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.843306    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gldn7\" (UniqueName: \"kubernetes.io/projected/4f6a5962-7e97-4b7d-af82-aa05f7d0785f-kube-api-access-gldn7\") pod \"payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl\" (UID: \"4f6a5962-7e97-4b7d-af82-aa05f7d0785f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl"
Oct 30 19:24:22.851015 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.843338    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fc8e7ff-e613-4074-825d-8b9da7356181-plugins-config-volume\") pod \"payload-processing-e2e-ait-250fb057-7598f9b64-d96bp\" (UID: \"3fc8e7ff-e613-4074-825d-8b9da7356181\") " pod="openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp"
Oct 30 19:24:22.851015 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.843374    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwqbt\" (UniqueName: \"kubernetes.io/projected/3fc8e7ff-e613-4074-825d-8b9da7356181-kube-api-access-lwqbt\") pod \"payload-processing-e2e-ait-250fb057-7598f9b64-d96bp\" (UID: \"3fc8e7ff-e613-4074-825d-8b9da7356181\") " pod="openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp"
Oct 30 19:24:22.851015 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.845162    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f6a5962-7e97-4b7d-af82-aa05f7d0785f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl\" (UID: \"4f6a5962-7e97-4b7d-af82-aa05f7d0785f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl"
Oct 30 19:24:22.867451 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.866842    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gldn7\" (UniqueName: \"kubernetes.io/projected/4f6a5962-7e97-4b7d-af82-aa05f7d0785f-kube-api-access-gldn7\") pod \"payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl\" (UID: \"4f6a5962-7e97-4b7d-af82-aa05f7d0785f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl"
Oct 30 19:24:22.943951 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.942464    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl"
Oct 30 19:24:22.962373 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.962331    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fc8e7ff-e613-4074-825d-8b9da7356181-plugins-config-volume\") pod \"payload-processing-e2e-ait-250fb057-7598f9b64-d96bp\" (UID: \"3fc8e7ff-e613-4074-825d-8b9da7356181\") " pod="openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp"
Oct 30 19:24:22.962637 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.962619    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwqbt\" (UniqueName: \"kubernetes.io/projected/3fc8e7ff-e613-4074-825d-8b9da7356181-kube-api-access-lwqbt\") pod \"payload-processing-e2e-ait-250fb057-7598f9b64-d96bp\" (UID: \"3fc8e7ff-e613-4074-825d-8b9da7356181\") " pod="openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp"
Oct 30 19:24:22.966580 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:22.966551    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fc8e7ff-e613-4074-825d-8b9da7356181-plugins-config-volume\") pod \"payload-processing-e2e-ait-250fb057-7598f9b64-d96bp\" (UID: \"3fc8e7ff-e613-4074-825d-8b9da7356181\") " pod="openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp"
Oct 30 19:24:23.021201 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:23.018456    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwqbt\" (UniqueName: \"kubernetes.io/projected/3fc8e7ff-e613-4074-825d-8b9da7356181-kube-api-access-lwqbt\") pod \"payload-processing-e2e-ait-250fb057-7598f9b64-d96bp\" (UID: \"3fc8e7ff-e613-4074-825d-8b9da7356181\") " pod="openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp"
Oct 30 19:24:23.021612 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:23.021583    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp"
Oct 30 19:24:23.577820 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:23.576286    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl"]
Oct 30 19:24:23.580715 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:24:23.580684    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f6a5962_7e97_4b7d_af82_aa05f7d0785f.slice/crio-b9d8fdcfac7fb137c77d1ee021b5cd3ea745377c9ab3e54c0c2276ead040b1ad WatchSource:0}: Error finding container b9d8fdcfac7fb137c77d1ee021b5cd3ea745377c9ab3e54c0c2276ead040b1ad: Status 404 returned error can't find the container with id b9d8fdcfac7fb137c77d1ee021b5cd3ea745377c9ab3e54c0c2276ead040b1ad
Oct 30 19:24:23.599866 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:23.599404    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp"]
Oct 30 19:24:23.612208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:23.612140    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8867-43cc7d"
Oct 30 19:24:23.699629 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:23.698971    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d3fbd80-1418-4464-943d-ea5a77165dc9-kube-api-access-8wpjj\" (UniqueName: \"kubernetes.io/projected/7d3fbd80-1418-4464-943d-ea5a77165dc9-kube-api-access-8wpjj\") pod \"7d3fbd80-1418-4464-943d-ea5a77165dc9\" (UID: \"7d3fbd80-1418-4464-943d-ea5a77165dc9\") "
Oct 30 19:24:23.701767 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:23.701736    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d3fbd80-1418-4464-943d-ea5a77165dc9-kube-api-access-8wpjj" pod "7d3fbd80-1418-4464-943d-ea5a77165dc9" (UID: "7d3fbd80-1418-4464-943d-ea5a77165dc9"). InnerVolumeSpecName "kube-api-access-8wpjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:23.801374 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:23.801335    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wpjj\" (UniqueName: \"kubernetes.io/projected/7d3fbd80-1418-4464-943d-ea5a77165dc9-kube-api-access-8wpjj\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:24.291248 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:24.282146    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp" event={"ID":"3fc8e7ff-e613-4074-825d-8b9da7356181","Type":"ContainerStarted","Data":"0f08fb9e26a489d4ed4be8cce841c68da0bbda0e2153d46ea8b6ffcda26ee4db"}
Oct 30 19:24:24.291248 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:24.282231    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp" event={"ID":"3fc8e7ff-e613-4074-825d-8b9da7356181","Type":"ContainerStarted","Data":"54ecf118f547bac5017a0533c3850040168300212c354f488a0814148b5499f6"}
Oct 30 19:24:24.291248 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:24.282273    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp"
Oct 30 19:24:24.291248 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:24.284291    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl" event={"ID":"4f6a5962-7e97-4b7d-af82-aa05f7d0785f","Type":"ContainerStarted","Data":"40917d91e256b667a2c056cdaccf1104ad7080e744a1494302578903c66fe68a"}
Oct 30 19:24:24.291248 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:24.284342    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl" event={"ID":"4f6a5962-7e97-4b7d-af82-aa05f7d0785f","Type":"ContainerStarted","Data":"b9d8fdcfac7fb137c77d1ee021b5cd3ea745377c9ab3e54c0c2276ead040b1ad"}
Oct 30 19:24:24.291248 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:24.285107    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl"
Oct 30 19:24:24.291248 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:24.287154    4524 generic.go:356] "Generic (PLEG): container finished" podID="7d3fbd80-1418-4464-943d-ea5a77165dc9" containerID="9663c224f64ab42c32515bd5ccfc8a82933901e793e1a24dabb66f28f6adeb2b" exitCode=137
Oct 30 19:24:24.291248 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:24.287313    4524 scope.go:122] "RemoveContainer" containerID="9663c224f64ab42c32515bd5ccfc8a82933901e793e1a24dabb66f28f6adeb2b"
Oct 30 19:24:24.291248 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:24.287407    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8867-43cc7d"
Oct 30 19:24:24.429321 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:24.428109    4524 scope.go:122] "RemoveContainer" containerID="9663c224f64ab42c32515bd5ccfc8a82933901e793e1a24dabb66f28f6adeb2b"
Oct 30 19:24:24.430464 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:24:24.430421    4524 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-8867-43cc7d\" is forbidden: User \"system:node:ip-10-0-137-200.us-east-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-200.us-east-2.compute.internal' and this object" podUID="7d3fbd80-1418-4464-943d-ea5a77165dc9" pod="opendatahub/test-curl-8867-43cc7d"
Oct 30 19:24:24.432536 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:24:24.432455    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9663c224f64ab42c32515bd5ccfc8a82933901e793e1a24dabb66f28f6adeb2b\": container with ID starting with 9663c224f64ab42c32515bd5ccfc8a82933901e793e1a24dabb66f28f6adeb2b not found: ID does not exist" containerID="9663c224f64ab42c32515bd5ccfc8a82933901e793e1a24dabb66f28f6adeb2b"
Oct 30 19:24:24.432536 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:24.432493    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9663c224f64ab42c32515bd5ccfc8a82933901e793e1a24dabb66f28f6adeb2b"} err="failed to get container status \"9663c224f64ab42c32515bd5ccfc8a82933901e793e1a24dabb66f28f6adeb2b\": rpc error: code = NotFound desc = could not find container \"9663c224f64ab42c32515bd5ccfc8a82933901e793e1a24dabb66f28f6adeb2b\": container with ID starting with 9663c224f64ab42c32515bd5ccfc8a82933901e793e1a24dabb66f28f6adeb2b not found: ID does not exist"
Oct 30 19:24:24.434532 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:24.434295    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl" podStartSLOduration=2.434280171 podStartE2EDuration="2.434280171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:24:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:24:24.428682457 +0000 UTC m=+1096.169673713" watchObservedRunningTime="2026-10-30 19:24:24.434280171 +0000 UTC m=+1096.175271246"
Oct 30 19:24:24.434640 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:24.434570    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d3fbd80-1418-4464-943d-ea5a77165dc9" path="/var/lib/kubelet/pods/7d3fbd80-1418-4464-943d-ea5a77165dc9/volumes"
Oct 30 19:24:24.439236 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:24.438404    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp" podStartSLOduration=2.4383922 podStartE2EDuration="2.4383922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:24:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:24:24.349925761 +0000 UTC m=+1096.090916840" watchObservedRunningTime="2026-10-30 19:24:24.4383922 +0000 UTC m=+1096.179383280"
Oct 30 19:24:24.514211 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:24:24.514121    4524 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d3fbd80_1418_4464_943d_ea5a77165dc9.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7d3fbd80_1418_4464_943d_ea5a77165dc9.slice/crio-59824d547a80e53bd93d76ea61366ed7372abbf84d303563754b5a5c6d7d7a19\": RecentStats: unable to find data in memory cache]"
Oct 30 19:24:25.153749 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.153715    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk"]
Oct 30 19:24:25.159529 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.159508    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk"
Oct 30 19:24:25.166878 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.166856    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-e308e736\""
Oct 30 19:24:25.167006 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.166965    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-e308e736-dockercfg-stk6l\""
Oct 30 19:24:25.191476 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.177530    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk"]
Oct 30 19:24:25.204698 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.204667    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt"]
Oct 30 19:24:25.209016 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.208995    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt"
Oct 30 19:24:25.221975 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.221951    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt"]
Oct 30 19:24:25.245810 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.245781    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44a66c02-a64d-4188-9cb0-db44dcb6db24-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk\" (UID: \"44a66c02-a64d-4188-9cb0-db44dcb6db24\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk"
Oct 30 19:24:25.246020 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.246000    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5rpf\" (UniqueName: \"kubernetes.io/projected/44a66c02-a64d-4188-9cb0-db44dcb6db24-kube-api-access-c5rpf\") pod \"payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk\" (UID: \"44a66c02-a64d-4188-9cb0-db44dcb6db24\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk"
Oct 30 19:24:25.350070 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.349114    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44a66c02-a64d-4188-9cb0-db44dcb6db24-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk\" (UID: \"44a66c02-a64d-4188-9cb0-db44dcb6db24\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk"
Oct 30 19:24:25.350070 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.349231    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e40817fb-5abe-4866-9bdf-92dd62c279a5-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt\" (UID: \"e40817fb-5abe-4866-9bdf-92dd62c279a5\") " pod="openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt"
Oct 30 19:24:25.350070 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.349348    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5rpf\" (UniqueName: \"kubernetes.io/projected/44a66c02-a64d-4188-9cb0-db44dcb6db24-kube-api-access-c5rpf\") pod \"payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk\" (UID: \"44a66c02-a64d-4188-9cb0-db44dcb6db24\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk"
Oct 30 19:24:25.350070 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.349426    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w45tk\" (UniqueName: \"kubernetes.io/projected/e40817fb-5abe-4866-9bdf-92dd62c279a5-kube-api-access-w45tk\") pod \"payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt\" (UID: \"e40817fb-5abe-4866-9bdf-92dd62c279a5\") " pod="openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt"
Oct 30 19:24:25.350445 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.350334    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44a66c02-a64d-4188-9cb0-db44dcb6db24-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk\" (UID: \"44a66c02-a64d-4188-9cb0-db44dcb6db24\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk"
Oct 30 19:24:25.401199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.400926    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5rpf\" (UniqueName: \"kubernetes.io/projected/44a66c02-a64d-4188-9cb0-db44dcb6db24-kube-api-access-c5rpf\") pod \"payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk\" (UID: \"44a66c02-a64d-4188-9cb0-db44dcb6db24\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk"
Oct 30 19:24:25.462264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.456118    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e40817fb-5abe-4866-9bdf-92dd62c279a5-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt\" (UID: \"e40817fb-5abe-4866-9bdf-92dd62c279a5\") " pod="openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt"
Oct 30 19:24:25.462264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.456366    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w45tk\" (UniqueName: \"kubernetes.io/projected/e40817fb-5abe-4866-9bdf-92dd62c279a5-kube-api-access-w45tk\") pod \"payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt\" (UID: \"e40817fb-5abe-4866-9bdf-92dd62c279a5\") " pod="openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt"
Oct 30 19:24:25.462264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.458114    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e40817fb-5abe-4866-9bdf-92dd62c279a5-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt\" (UID: \"e40817fb-5abe-4866-9bdf-92dd62c279a5\") " pod="openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt"
Oct 30 19:24:25.483193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.476981    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w45tk\" (UniqueName: \"kubernetes.io/projected/e40817fb-5abe-4866-9bdf-92dd62c279a5-kube-api-access-w45tk\") pod \"payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt\" (UID: \"e40817fb-5abe-4866-9bdf-92dd62c279a5\") " pod="openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt"
Oct 30 19:24:25.508230 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.501490    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk"
Oct 30 19:24:25.549204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:25.543871    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt"
Oct 30 19:24:27.858246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:27.823453    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk"]
Oct 30 19:24:28.208213 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:28.201998    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt"]
Oct 30 19:24:29.093738 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:29.093691    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz" podUID="e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:24:29.100009 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:29.099974    4524 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz" podUID="e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:24:29.433282 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:29.399511    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:24:29.433282 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:29.399582    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp" podUID="e6dd71b4-1741-4da3-99f4-82121e585d89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:24:29.446195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:29.434151    4524 patch_prober.go:29] interesting pod/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:24:29.450193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:29.446469    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6" podUID="75f42a71-342e-4c05-80c7-a1e7b6336984" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:24:29.771709 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:29.771614    4524 patch_prober.go:29] interesting pod/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:24:29.771993 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:29.771960    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd" podUID="e53dd061-8aa9-4ace-9140-aea8a7375c20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:24:30.178192 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:24:30.160510    4524 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.416s"
Oct 30 19:24:30.178192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:30.160563    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk" event={"ID":"44a66c02-a64d-4188-9cb0-db44dcb6db24","Type":"ContainerStarted","Data":"7e6aeb560dccb01009db78b9511f8f95807e651a4b47288f69f9842575ffda45"}
Oct 30 19:24:30.178192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:30.160625    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt" event={"ID":"e40817fb-5abe-4866-9bdf-92dd62c279a5","Type":"ContainerStarted","Data":"c55dad6a0e81e1d6636406f82d85f2a25d15f729358088edac4faafd9a0a7988"}
Oct 30 19:24:30.757194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:30.753034    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz" podUID="e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:24:30.794390 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:30.794293    4524 patch_prober.go:29] interesting pod/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:24:30.807114 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:30.807076    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9" podUID="83f517e5-9b8e-425a-845b-56253704cd78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:24:30.874304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:30.860905    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:24:30.874304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:30.860967    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf" podUID="c0210704-8d10-4dad-8f61-7e028f2bbab8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:24:30.874304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:30.866740    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:24:30.893328 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:30.885664    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:24:31.003498 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:30.899743    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc" podUID="d72b215c-1d3b-400c-9238-906b0199f4c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:24:31.003498 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:30.999359    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:24:31.003498 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:30.999652    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4" podUID="3b269ee8-7299-4e79-8f77-ed5e4e76730d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:24:31.223400 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:31.217278    4524 patch_prober.go:29] interesting pod/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:24:31.223400 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:31.217344    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6" podUID="1d63b36f-3cfa-4438-8fb1-21480c9e9527" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:24:31.613651 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:31.609111    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk"
Oct 30 19:24:31.747519 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:31.744730    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:24:31.776684 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:31.776639    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:24:31.776948 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:31.776923    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk" podUID="a4fd9180-f4aa-4986-b2f0-955296fe17ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:24:32.094245 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:32.068331    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk" podStartSLOduration=7.068310874 podStartE2EDuration="7.068310874s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:24:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:24:31.776295603 +0000 UTC m=+1103.517286683" watchObservedRunningTime="2026-10-30 19:24:32.068310874 +0000 UTC m=+1103.809301955"
Oct 30 19:24:32.629978 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:32.625353    4524 patch_prober.go:29] interesting pod/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:24:32.629978 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:32.625408    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n" podUID="555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:24:32.883602 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:32.883503    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk" event={"ID":"44a66c02-a64d-4188-9cb0-db44dcb6db24","Type":"ContainerStarted","Data":"61548b7fd8a2aef82316744affa046981c92cb3d572859e6a7f32d19a45b0d7c"}
Oct 30 19:24:32.956872 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:32.956832    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt" event={"ID":"e40817fb-5abe-4866-9bdf-92dd62c279a5","Type":"ContainerStarted","Data":"0db07c730973917f4c312d9c4bc311a4ca23c588cf704868f71e1c409e74ba90"}
Oct 30 19:24:32.958133 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:32.958105    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt"
Oct 30 19:24:33.237252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:33.236376    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt" podStartSLOduration=8.23630866 podStartE2EDuration="8.23630866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:24:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:24:33.055592729 +0000 UTC m=+1104.796583809" watchObservedRunningTime="2026-10-30 19:24:33.23630866 +0000 UTC m=+1104.977299750"
Oct 30 19:24:35.303678 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:35.303647    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp"
Oct 30 19:24:35.635314 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:35.635280    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"]
Oct 30 19:24:35.641245 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:35.640277    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"
Oct 30 19:24:35.651935 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:35.644974    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-250fb057-serving-cert\""
Oct 30 19:24:35.651935 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:35.651856    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"]
Oct 30 19:24:35.789217 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:35.789135    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e21b01f-1165-4487-b297-18d11d747650-openshift-service-ca\") pod \"maas-api-e2e-ait-250fb057-6d875896d5-q4q5q\" (UID: \"5e21b01f-1165-4487-b297-18d11d747650\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"
Oct 30 19:24:35.789449 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:35.789397    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkqp9\" (UniqueName: \"kubernetes.io/projected/5e21b01f-1165-4487-b297-18d11d747650-kube-api-access-gkqp9\") pod \"maas-api-e2e-ait-250fb057-6d875896d5-q4q5q\" (UID: \"5e21b01f-1165-4487-b297-18d11d747650\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"
Oct 30 19:24:35.789545 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:35.789475    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e21b01f-1165-4487-b297-18d11d747650-maas-api-tls\") pod \"maas-api-e2e-ait-250fb057-6d875896d5-q4q5q\" (UID: \"5e21b01f-1165-4487-b297-18d11d747650\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"
Oct 30 19:24:35.789610 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:35.789564    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e21b01f-1165-4487-b297-18d11d747650-metrics-certs\") pod \"maas-api-e2e-ait-250fb057-6d875896d5-q4q5q\" (UID: \"5e21b01f-1165-4487-b297-18d11d747650\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"
Oct 30 19:24:35.902373 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:35.900916    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e21b01f-1165-4487-b297-18d11d747650-metrics-certs\") pod \"maas-api-e2e-ait-250fb057-6d875896d5-q4q5q\" (UID: \"5e21b01f-1165-4487-b297-18d11d747650\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"
Oct 30 19:24:35.902373 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:35.900993    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e21b01f-1165-4487-b297-18d11d747650-openshift-service-ca\") pod \"maas-api-e2e-ait-250fb057-6d875896d5-q4q5q\" (UID: \"5e21b01f-1165-4487-b297-18d11d747650\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"
Oct 30 19:24:35.902373 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:35.901227    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkqp9\" (UniqueName: \"kubernetes.io/projected/5e21b01f-1165-4487-b297-18d11d747650-kube-api-access-gkqp9\") pod \"maas-api-e2e-ait-250fb057-6d875896d5-q4q5q\" (UID: \"5e21b01f-1165-4487-b297-18d11d747650\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"
Oct 30 19:24:35.902373 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:35.901300    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e21b01f-1165-4487-b297-18d11d747650-maas-api-tls\") pod \"maas-api-e2e-ait-250fb057-6d875896d5-q4q5q\" (UID: \"5e21b01f-1165-4487-b297-18d11d747650\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"
Oct 30 19:24:35.902373 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:24:35.901488    4524 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-serving-cert: secret "maas-api-e2e-ait-250fb057-serving-cert" not found
Oct 30 19:24:35.902373 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:24:35.901558    4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e21b01f-1165-4487-b297-18d11d747650-maas-api-tls podName:5e21b01f-1165-4487-b297-18d11d747650 nodeName:}" failed. No retries permitted until 2026-10-30 19:24:36.401535995 +0000 UTC m=+1108.142527065 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5e21b01f-1165-4487-b297-18d11d747650-maas-api-tls") pod "maas-api-e2e-ait-250fb057-6d875896d5-q4q5q" (UID: "5e21b01f-1165-4487-b297-18d11d747650") : secret "maas-api-e2e-ait-250fb057-serving-cert" not found
Oct 30 19:24:35.911507 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:35.911472    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e21b01f-1165-4487-b297-18d11d747650-openshift-service-ca\") pod \"maas-api-e2e-ait-250fb057-6d875896d5-q4q5q\" (UID: \"5e21b01f-1165-4487-b297-18d11d747650\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"
Oct 30 19:24:35.925230 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:35.925075    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkqp9\" (UniqueName: \"kubernetes.io/projected/5e21b01f-1165-4487-b297-18d11d747650-kube-api-access-gkqp9\") pod \"maas-api-e2e-ait-250fb057-6d875896d5-q4q5q\" (UID: \"5e21b01f-1165-4487-b297-18d11d747650\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"
Oct 30 19:24:35.929324 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:35.929259    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e21b01f-1165-4487-b297-18d11d747650-metrics-certs\") pod \"maas-api-e2e-ait-250fb057-6d875896d5-q4q5q\" (UID: \"5e21b01f-1165-4487-b297-18d11d747650\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"
Oct 30 19:24:36.473898 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:36.473867    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl"
Oct 30 19:24:36.483937 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:36.483859    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e21b01f-1165-4487-b297-18d11d747650-maas-api-tls\") pod \"maas-api-e2e-ait-250fb057-6d875896d5-q4q5q\" (UID: \"5e21b01f-1165-4487-b297-18d11d747650\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"
Oct 30 19:24:36.559865 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:36.559829    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e21b01f-1165-4487-b297-18d11d747650-maas-api-tls\") pod \"maas-api-e2e-ait-250fb057-6d875896d5-q4q5q\" (UID: \"5e21b01f-1165-4487-b297-18d11d747650\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"
Oct 30 19:24:36.569741 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:36.569686    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"
Oct 30 19:24:36.957575 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:36.952542    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"]
Oct 30 19:24:36.957575 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:24:36.955124    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e21b01f_1165_4487_b297_18d11d747650.slice/crio-8481d5eb6fe247fdedbfadb2aeb844aff8c3ed5ca83c009439d2f9a5a15ced44 WatchSource:0}: Error finding container 8481d5eb6fe247fdedbfadb2aeb844aff8c3ed5ca83c009439d2f9a5a15ced44: Status 404 returned error can't find the container with id 8481d5eb6fe247fdedbfadb2aeb844aff8c3ed5ca83c009439d2f9a5a15ced44
Oct 30 19:24:36.994101 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:36.993377    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q" event={"ID":"5e21b01f-1165-4487-b297-18d11d747650","Type":"ContainerStarted","Data":"8481d5eb6fe247fdedbfadb2aeb844aff8c3ed5ca83c009439d2f9a5a15ced44"}
Oct 30 19:24:38.001964 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:38.001927    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q" event={"ID":"5e21b01f-1165-4487-b297-18d11d747650","Type":"ContainerStarted","Data":"c0b406bb7da0c41089b7dc06d25e6848809784229b4b7a4ada36bd658d7e19b8"}
Oct 30 19:24:38.002465 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:38.002156    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"
Oct 30 19:24:38.024518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:38.024464    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q" podStartSLOduration=2.473940557 podStartE2EDuration="3.024449809s" totalImagesPullingTime="550.509252ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:24:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:24:38.020302338 +0000 UTC m=+1109.761293418" watchObservedRunningTime="2026-10-30 19:24:38.024449809 +0000 UTC m=+1109.765440938"
Oct 30 19:24:39.141821 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:39.138902    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"]
Oct 30 19:24:39.144880 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:39.144816    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"
Oct 30 19:24:39.161716 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:39.156753    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"]
Oct 30 19:24:39.178277 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:39.175248    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-e308e736-serving-cert\""
Oct 30 19:24:39.193072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:39.193036    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa344907-7495-4eba-811e-e62153b9c8d8-maas-api-tls\") pod \"maas-api-e2e-migrate-e308e736-58444d988d-zx6tn\" (UID: \"aa344907-7495-4eba-811e-e62153b9c8d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"
Oct 30 19:24:39.193297 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:39.193279    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa344907-7495-4eba-811e-e62153b9c8d8-openshift-service-ca\") pod \"maas-api-e2e-migrate-e308e736-58444d988d-zx6tn\" (UID: \"aa344907-7495-4eba-811e-e62153b9c8d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"
Oct 30 19:24:39.193388 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:39.193315    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa344907-7495-4eba-811e-e62153b9c8d8-metrics-certs\") pod \"maas-api-e2e-migrate-e308e736-58444d988d-zx6tn\" (UID: \"aa344907-7495-4eba-811e-e62153b9c8d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"
Oct 30 19:24:39.193445 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:39.193410    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7cpt\" (UniqueName: \"kubernetes.io/projected/aa344907-7495-4eba-811e-e62153b9c8d8-kube-api-access-m7cpt\") pod \"maas-api-e2e-migrate-e308e736-58444d988d-zx6tn\" (UID: \"aa344907-7495-4eba-811e-e62153b9c8d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"
Oct 30 19:24:39.311905 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:39.310340    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa344907-7495-4eba-811e-e62153b9c8d8-openshift-service-ca\") pod \"maas-api-e2e-migrate-e308e736-58444d988d-zx6tn\" (UID: \"aa344907-7495-4eba-811e-e62153b9c8d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"
Oct 30 19:24:39.311905 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:39.310404    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa344907-7495-4eba-811e-e62153b9c8d8-metrics-certs\") pod \"maas-api-e2e-migrate-e308e736-58444d988d-zx6tn\" (UID: \"aa344907-7495-4eba-811e-e62153b9c8d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"
Oct 30 19:24:39.311905 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:39.310483    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7cpt\" (UniqueName: \"kubernetes.io/projected/aa344907-7495-4eba-811e-e62153b9c8d8-kube-api-access-m7cpt\") pod \"maas-api-e2e-migrate-e308e736-58444d988d-zx6tn\" (UID: \"aa344907-7495-4eba-811e-e62153b9c8d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"
Oct 30 19:24:39.311905 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:39.310691    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa344907-7495-4eba-811e-e62153b9c8d8-maas-api-tls\") pod \"maas-api-e2e-migrate-e308e736-58444d988d-zx6tn\" (UID: \"aa344907-7495-4eba-811e-e62153b9c8d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"
Oct 30 19:24:39.345509 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:39.345425    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa344907-7495-4eba-811e-e62153b9c8d8-openshift-service-ca\") pod \"maas-api-e2e-migrate-e308e736-58444d988d-zx6tn\" (UID: \"aa344907-7495-4eba-811e-e62153b9c8d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"
Oct 30 19:24:39.354441 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:39.354409    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa344907-7495-4eba-811e-e62153b9c8d8-metrics-certs\") pod \"maas-api-e2e-migrate-e308e736-58444d988d-zx6tn\" (UID: \"aa344907-7495-4eba-811e-e62153b9c8d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"
Oct 30 19:24:39.354605 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:39.354409    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa344907-7495-4eba-811e-e62153b9c8d8-maas-api-tls\") pod \"maas-api-e2e-migrate-e308e736-58444d988d-zx6tn\" (UID: \"aa344907-7495-4eba-811e-e62153b9c8d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"
Oct 30 19:24:39.401329 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:39.401221    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7cpt\" (UniqueName: \"kubernetes.io/projected/aa344907-7495-4eba-811e-e62153b9c8d8-kube-api-access-m7cpt\") pod \"maas-api-e2e-migrate-e308e736-58444d988d-zx6tn\" (UID: \"aa344907-7495-4eba-811e-e62153b9c8d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"
Oct 30 19:24:39.490574 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:39.490525    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"
Oct 30 19:24:39.798156 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:39.798076    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"]
Oct 30 19:24:39.801983 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:24:39.801955    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa344907_7495_4eba_811e_e62153b9c8d8.slice/crio-40dea5f76f476d5b56f25d7ae0e3078d70cb83ac438cd6629f63965cef371861 WatchSource:0}: Error finding container 40dea5f76f476d5b56f25d7ae0e3078d70cb83ac438cd6629f63965cef371861: Status 404 returned error can't find the container with id 40dea5f76f476d5b56f25d7ae0e3078d70cb83ac438cd6629f63965cef371861
Oct 30 19:24:40.021795 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:40.021747    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn" event={"ID":"aa344907-7495-4eba-811e-e62153b9c8d8","Type":"ContainerStarted","Data":"40dea5f76f476d5b56f25d7ae0e3078d70cb83ac438cd6629f63965cef371861"}
Oct 30 19:24:41.032363 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:41.032327    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn" event={"ID":"aa344907-7495-4eba-811e-e62153b9c8d8","Type":"ContainerStarted","Data":"2439d604753e3a1b795d8ad116b8efe7c569d12f294ba8f0df1da5cc2805f145"}
Oct 30 19:24:41.032765 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:41.032424    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"
Oct 30 19:24:41.059874 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:41.059807    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn" podStartSLOduration=1.525319043 podStartE2EDuration="2.05978953s" totalImagesPullingTime="534.470487ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:24:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:24:41.059030608 +0000 UTC m=+1112.800021688" watchObservedRunningTime="2026-10-30 19:24:41.05978953 +0000 UTC m=+1112.800780600"
Oct 30 19:24:43.970143 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:43.970110    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk"
Oct 30 19:24:44.020469 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:44.020434    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"
Oct 30 19:24:44.928100 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:44.928062    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"]
Oct 30 19:24:44.933127 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:44.933103    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:44.939142 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:44.939116    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-bce71f\"/\"kube-root-ca.crt\""
Oct 30 19:24:44.939626 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:44.939516    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-bce71f\"/\"default-dockercfg-rgxzp\""
Oct 30 19:24:44.939747 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:44.939580    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-bce71f\"/\"openshift-service-ca.crt\""
Oct 30 19:24:44.940159 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:44.940133    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-bce71f\"/\"distinct-w6-bce71f-kserve-self-signed-certs\""
Oct 30 19:24:44.948158 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:44.948135    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"]
Oct 30 19:24:44.977910 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:44.977878    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt"
Oct 30 19:24:45.068402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:45.068366    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-model-cache\") pod \"distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:45.068809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:45.068467    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-dshm\") pod \"distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:45.068809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:45.068542    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qttsl\" (UniqueName: \"kubernetes.io/projected/24fbbf1e-3438-4436-b4cd-9890fe28676a-kube-api-access-qttsl\") pod \"distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:45.068809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:45.068620    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-tmp-dir\") pod \"distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:45.068809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:45.068696    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24fbbf1e-3438-4436-b4cd-9890fe28676a-tls-certs\") pod \"distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:45.068809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:45.068760    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-home\") pod \"distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:45.172134 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:45.172079    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qttsl\" (UniqueName: \"kubernetes.io/projected/24fbbf1e-3438-4436-b4cd-9890fe28676a-kube-api-access-qttsl\") pod \"distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:45.180541 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:45.180446    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-tmp-dir\") pod \"distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:45.209380 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:45.209336    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24fbbf1e-3438-4436-b4cd-9890fe28676a-tls-certs\") pod \"distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:45.211505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:45.211477    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-home\") pod \"distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:45.212335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:45.181335    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-tmp-dir\") pod \"distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:45.212335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:45.211888    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-model-cache\") pod \"distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:45.212335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:45.211985    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-home\") pod \"distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:45.212335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:45.211989    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-dshm\") pod \"distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:45.212606 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:45.212431    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-model-cache\") pod \"distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:45.233626 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:45.229879    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qttsl\" (UniqueName: \"kubernetes.io/projected/24fbbf1e-3438-4436-b4cd-9890fe28676a-kube-api-access-qttsl\") pod \"distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:45.233626 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:45.230754    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-dshm\") pod \"distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:45.234498 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:45.234471    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24fbbf1e-3438-4436-b4cd-9890fe28676a-tls-certs\") pod \"distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:45.261016 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:45.260910    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:45.630414 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:24:45.630380    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:24:45.670435 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:45.670402    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"]
Oct 30 19:24:45.670786 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:24:45.670762    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24fbbf1e_3438_4436_b4cd_9890fe28676a.slice/crio-c39cb6422f0326b587cadedcb58740fdd461424c8c1e31d84176cfc8007f5c35 WatchSource:0}: Error finding container c39cb6422f0326b587cadedcb58740fdd461424c8c1e31d84176cfc8007f5c35: Status 404 returned error can't find the container with id c39cb6422f0326b587cadedcb58740fdd461424c8c1e31d84176cfc8007f5c35
Oct 30 19:24:46.102189 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:46.101803    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j" event={"ID":"24fbbf1e-3438-4436-b4cd-9890fe28676a","Type":"ContainerStarted","Data":"6bbfebe44145e43886a4a87389b690a9fdb123417773335949ed9932b46cabbe"}
Oct 30 19:24:46.102189 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:46.101868    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j" event={"ID":"24fbbf1e-3438-4436-b4cd-9890fe28676a","Type":"ContainerStarted","Data":"c39cb6422f0326b587cadedcb58740fdd461424c8c1e31d84176cfc8007f5c35"}
Oct 30 19:24:46.121656 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:46.121604    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j" podStartSLOduration=2.121588639 podStartE2EDuration="2.121588639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:24:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:24:46.120006825 +0000 UTC m=+1117.860997904" watchObservedRunningTime="2026-10-30 19:24:46.121588639 +0000 UTC m=+1117.862579718"
Oct 30 19:24:46.261663 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:46.261624    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:47.044725 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:47.044698    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"
Oct 30 19:24:49.046236 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:49.046196    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc"]
Oct 30 19:24:49.050964 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:49.050940    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc"
Oct 30 19:24:49.059857 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:49.059826    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-7c74535f\""
Oct 30 19:24:49.062428 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:49.062394    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-7c74535f-dockercfg-4tfmz\""
Oct 30 19:24:49.065321 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:49.065295    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc"]
Oct 30 19:24:49.179563 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:49.170638    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn9hz\" (UniqueName: \"kubernetes.io/projected/c57d766d-cf24-4997-aeca-810860a18ca1-kube-api-access-sn9hz\") pod \"payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc\" (UID: \"c57d766d-cf24-4997-aeca-810860a18ca1\") " pod="openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc"
Oct 30 19:24:49.180257 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:49.180217    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c57d766d-cf24-4997-aeca-810860a18ca1-plugins-config-volume\") pod \"payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc\" (UID: \"c57d766d-cf24-4997-aeca-810860a18ca1\") " pod="openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc"
Oct 30 19:24:49.307578 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:49.307486    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn9hz\" (UniqueName: \"kubernetes.io/projected/c57d766d-cf24-4997-aeca-810860a18ca1-kube-api-access-sn9hz\") pod \"payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc\" (UID: \"c57d766d-cf24-4997-aeca-810860a18ca1\") " pod="openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc"
Oct 30 19:24:49.307902 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:49.307878    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c57d766d-cf24-4997-aeca-810860a18ca1-plugins-config-volume\") pod \"payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc\" (UID: \"c57d766d-cf24-4997-aeca-810860a18ca1\") " pod="openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc"
Oct 30 19:24:49.309057 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:49.309034    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c57d766d-cf24-4997-aeca-810860a18ca1-plugins-config-volume\") pod \"payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc\" (UID: \"c57d766d-cf24-4997-aeca-810860a18ca1\") " pod="openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc"
Oct 30 19:24:49.352994 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:49.352960    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn9hz\" (UniqueName: \"kubernetes.io/projected/c57d766d-cf24-4997-aeca-810860a18ca1-kube-api-access-sn9hz\") pod \"payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc\" (UID: \"c57d766d-cf24-4997-aeca-810860a18ca1\") " pod="openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc"
Oct 30 19:24:49.392624 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:49.392591    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc"
Oct 30 19:24:49.784091 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:49.784039    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc"]
Oct 30 19:24:49.790289 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:24:49.788024    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc57d766d_cf24_4997_aeca_810860a18ca1.slice/crio-15fdbba332957b0bb677a3461e08d8b67d49df23277b415fd907d091c076ae28 WatchSource:0}: Error finding container 15fdbba332957b0bb677a3461e08d8b67d49df23277b415fd907d091c076ae28: Status 404 returned error can't find the container with id 15fdbba332957b0bb677a3461e08d8b67d49df23277b415fd907d091c076ae28
Oct 30 19:24:50.145470 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:50.145430    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc" event={"ID":"c57d766d-cf24-4997-aeca-810860a18ca1","Type":"ContainerStarted","Data":"a714d925a0c9e5de58f8ecaa73164a76a8ffa78df36782ccbfe5cff0c60f4d5e"}
Oct 30 19:24:50.145877 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:50.145516    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc" event={"ID":"c57d766d-cf24-4997-aeca-810860a18ca1","Type":"ContainerStarted","Data":"15fdbba332957b0bb677a3461e08d8b67d49df23277b415fd907d091c076ae28"}
Oct 30 19:24:50.145877 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:50.145639    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc"
Oct 30 19:24:50.166348 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:50.166302    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc" podStartSLOduration=1.166288036 podStartE2EDuration="1.166288036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:24:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:24:50.163049298 +0000 UTC m=+1121.904040377" watchObservedRunningTime="2026-10-30 19:24:50.166288036 +0000 UTC m=+1121.907279116"
Oct 30 19:24:55.263566 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:55.263521    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:55.434297 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:55.434263    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:55.930655 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:55.930624    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:24:57.660332 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:57.660295    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp"]
Oct 30 19:24:57.665728 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:57.665692    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl"]
Oct 30 19:24:59.446670 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:59.446636    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"]
Oct 30 19:24:59.492945 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:59.492847    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"]
Oct 30 19:24:59.493305 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:59.493245    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"
Oct 30 19:24:59.497749 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:59.497721    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-7c74535f-serving-cert\""
Oct 30 19:24:59.532913 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:59.532875    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/445c0240-a758-4f58-80ac-9c99afc0a2d6-metrics-certs\") pod \"maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p\" (UID: \"445c0240-a758-4f58-80ac-9c99afc0a2d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"
Oct 30 19:24:59.533082 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:59.532969    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/445c0240-a758-4f58-80ac-9c99afc0a2d6-openshift-service-ca\") pod \"maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p\" (UID: \"445c0240-a758-4f58-80ac-9c99afc0a2d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"
Oct 30 19:24:59.533082 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:59.533035    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/445c0240-a758-4f58-80ac-9c99afc0a2d6-maas-api-tls\") pod \"maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p\" (UID: \"445c0240-a758-4f58-80ac-9c99afc0a2d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"
Oct 30 19:24:59.533252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:59.533230    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxvzn\" (UniqueName: \"kubernetes.io/projected/445c0240-a758-4f58-80ac-9c99afc0a2d6-kube-api-access-qxvzn\") pod \"maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p\" (UID: \"445c0240-a758-4f58-80ac-9c99afc0a2d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"
Oct 30 19:24:59.645204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:59.635997    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/445c0240-a758-4f58-80ac-9c99afc0a2d6-metrics-certs\") pod \"maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p\" (UID: \"445c0240-a758-4f58-80ac-9c99afc0a2d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"
Oct 30 19:24:59.645204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:59.636066    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/445c0240-a758-4f58-80ac-9c99afc0a2d6-openshift-service-ca\") pod \"maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p\" (UID: \"445c0240-a758-4f58-80ac-9c99afc0a2d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"
Oct 30 19:24:59.645204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:59.636158    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/445c0240-a758-4f58-80ac-9c99afc0a2d6-maas-api-tls\") pod \"maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p\" (UID: \"445c0240-a758-4f58-80ac-9c99afc0a2d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"
Oct 30 19:24:59.645204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:59.636250    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxvzn\" (UniqueName: \"kubernetes.io/projected/445c0240-a758-4f58-80ac-9c99afc0a2d6-kube-api-access-qxvzn\") pod \"maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p\" (UID: \"445c0240-a758-4f58-80ac-9c99afc0a2d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"
Oct 30 19:24:59.645204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:59.641007    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/445c0240-a758-4f58-80ac-9c99afc0a2d6-openshift-service-ca\") pod \"maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p\" (UID: \"445c0240-a758-4f58-80ac-9c99afc0a2d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"
Oct 30 19:24:59.652968 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:59.652933    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/445c0240-a758-4f58-80ac-9c99afc0a2d6-maas-api-tls\") pod \"maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p\" (UID: \"445c0240-a758-4f58-80ac-9c99afc0a2d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"
Oct 30 19:24:59.660294 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:59.653965    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxvzn\" (UniqueName: \"kubernetes.io/projected/445c0240-a758-4f58-80ac-9c99afc0a2d6-kube-api-access-qxvzn\") pod \"maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p\" (UID: \"445c0240-a758-4f58-80ac-9c99afc0a2d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"
Oct 30 19:24:59.664957 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:59.664926    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/445c0240-a758-4f58-80ac-9c99afc0a2d6-metrics-certs\") pod \"maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p\" (UID: \"445c0240-a758-4f58-80ac-9c99afc0a2d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"
Oct 30 19:24:59.815268 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:24:59.815143    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"
Oct 30 19:25:00.164233 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:00.155629    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"]
Oct 30 19:25:00.927753 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:00.927714    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p" event={"ID":"445c0240-a758-4f58-80ac-9c99afc0a2d6","Type":"ContainerStarted","Data":"6305cfba466c3695801614872480b88a157cae8714d833d29b02bda0860159f1"}
Oct 30 19:25:00.928164 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:00.927786    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p" event={"ID":"445c0240-a758-4f58-80ac-9c99afc0a2d6","Type":"ContainerStarted","Data":"5551b80f9f7bedfad33177e086f47e5f54769926e4c150d74e5fab88e2848a98"}
Oct 30 19:25:00.928164 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:00.927824    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"
Oct 30 19:25:00.951084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:00.951032    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p" podStartSLOduration=1.3719675599999999 podStartE2EDuration="1.951014225s" totalImagesPullingTime="579.046665ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:00.947231121 +0000 UTC m=+1132.688222209" watchObservedRunningTime="2026-10-30 19:25:00.951014225 +0000 UTC m=+1132.692005306"
Oct 30 19:25:01.250787 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:01.250757    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc"
Oct 30 19:25:02.661241 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:02.661188    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp" podUID="3fc8e7ff-e613-4074-825d-8b9da7356181" containerName="payload-processing" containerID="cri-o://0f08fb9e26a489d4ed4be8cce841c68da0bbda0e2153d46ea8b6ffcda26ee4db" gracePeriod=25
Oct 30 19:25:02.666790 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:02.666734    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl" podUID="4f6a5962-7e97-4b7d-af82-aa05f7d0785f" containerName="payload-pre-processing" containerID="cri-o://40917d91e256b667a2c056cdaccf1104ad7080e744a1494302578903c66fe68a" gracePeriod=25
Oct 30 19:25:03.102507 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:03.102466    4524 generic.go:356] "Generic (PLEG): container finished" podID="4f6a5962-7e97-4b7d-af82-aa05f7d0785f" containerID="40917d91e256b667a2c056cdaccf1104ad7080e744a1494302578903c66fe68a" exitCode=0
Oct 30 19:25:03.102678 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:03.102650    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl" event={"ID":"4f6a5962-7e97-4b7d-af82-aa05f7d0785f","Type":"ContainerDied","Data":"40917d91e256b667a2c056cdaccf1104ad7080e744a1494302578903c66fe68a"}
Oct 30 19:25:03.104251 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:03.104215    4524 generic.go:356] "Generic (PLEG): container finished" podID="3fc8e7ff-e613-4074-825d-8b9da7356181" containerID="0f08fb9e26a489d4ed4be8cce841c68da0bbda0e2153d46ea8b6ffcda26ee4db" exitCode=0
Oct 30 19:25:03.104394 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:03.104279    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp" event={"ID":"3fc8e7ff-e613-4074-825d-8b9da7356181","Type":"ContainerDied","Data":"0f08fb9e26a489d4ed4be8cce841c68da0bbda0e2153d46ea8b6ffcda26ee4db"}
Oct 30 19:25:03.111381 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:03.111362    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp"
Oct 30 19:25:03.115706 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:03.115685    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl"
Oct 30 19:25:03.203564 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:03.203528    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3fc8e7ff-e613-4074-825d-8b9da7356181-kube-api-access-lwqbt\" (UniqueName: \"kubernetes.io/projected/3fc8e7ff-e613-4074-825d-8b9da7356181-kube-api-access-lwqbt\") pod \"3fc8e7ff-e613-4074-825d-8b9da7356181\" (UID: \"3fc8e7ff-e613-4074-825d-8b9da7356181\") "
Oct 30 19:25:03.203731 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:03.203629    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4f6a5962-7e97-4b7d-af82-aa05f7d0785f-kube-api-access-gldn7\" (UniqueName: \"kubernetes.io/projected/4f6a5962-7e97-4b7d-af82-aa05f7d0785f-kube-api-access-gldn7\") pod \"4f6a5962-7e97-4b7d-af82-aa05f7d0785f\" (UID: \"4f6a5962-7e97-4b7d-af82-aa05f7d0785f\") "
Oct 30 19:25:03.203832 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:03.203813    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3fc8e7ff-e613-4074-825d-8b9da7356181-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fc8e7ff-e613-4074-825d-8b9da7356181-plugins-config-volume\") pod \"3fc8e7ff-e613-4074-825d-8b9da7356181\" (UID: \"3fc8e7ff-e613-4074-825d-8b9da7356181\") "
Oct 30 19:25:03.203906 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:03.203889    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4f6a5962-7e97-4b7d-af82-aa05f7d0785f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f6a5962-7e97-4b7d-af82-aa05f7d0785f-plugins-config-volume\") pod \"4f6a5962-7e97-4b7d-af82-aa05f7d0785f\" (UID: \"4f6a5962-7e97-4b7d-af82-aa05f7d0785f\") "
Oct 30 19:25:03.204205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:03.204158    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fc8e7ff-e613-4074-825d-8b9da7356181-plugins-config-volume" pod "3fc8e7ff-e613-4074-825d-8b9da7356181" (UID: "3fc8e7ff-e613-4074-825d-8b9da7356181"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:25:03.204507 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:03.204364    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f6a5962-7e97-4b7d-af82-aa05f7d0785f-plugins-config-volume" pod "4f6a5962-7e97-4b7d-af82-aa05f7d0785f" (UID: "4f6a5962-7e97-4b7d-af82-aa05f7d0785f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:25:03.204660 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:03.204633    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3fc8e7ff-e613-4074-825d-8b9da7356181-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:03.204787 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:03.204663    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4f6a5962-7e97-4b7d-af82-aa05f7d0785f-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:03.206231 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:03.206200    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fc8e7ff-e613-4074-825d-8b9da7356181-kube-api-access-lwqbt" pod "3fc8e7ff-e613-4074-825d-8b9da7356181" (UID: "3fc8e7ff-e613-4074-825d-8b9da7356181"). InnerVolumeSpecName "kube-api-access-lwqbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:25:03.206339 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:03.206281    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f6a5962-7e97-4b7d-af82-aa05f7d0785f-kube-api-access-gldn7" pod "4f6a5962-7e97-4b7d-af82-aa05f7d0785f" (UID: "4f6a5962-7e97-4b7d-af82-aa05f7d0785f"). InnerVolumeSpecName "kube-api-access-gldn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:25:03.306704 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:03.306665    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gldn7\" (UniqueName: \"kubernetes.io/projected/4f6a5962-7e97-4b7d-af82-aa05f7d0785f-kube-api-access-gldn7\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:03.306704 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:03.306703    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwqbt\" (UniqueName: \"kubernetes.io/projected/3fc8e7ff-e613-4074-825d-8b9da7356181-kube-api-access-lwqbt\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:04.114047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:04.114017    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp"
Oct 30 19:25:04.114565 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:04.114051    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp" event={"ID":"3fc8e7ff-e613-4074-825d-8b9da7356181","Type":"ContainerDied","Data":"54ecf118f547bac5017a0533c3850040168300212c354f488a0814148b5499f6"}
Oct 30 19:25:04.114565 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:04.114180    4524 scope.go:122] "RemoveContainer" containerID="0f08fb9e26a489d4ed4be8cce841c68da0bbda0e2153d46ea8b6ffcda26ee4db"
Oct 30 19:25:04.116412 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:04.116394    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl"
Oct 30 19:25:04.116537 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:04.116446    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl" event={"ID":"4f6a5962-7e97-4b7d-af82-aa05f7d0785f","Type":"ContainerDied","Data":"b9d8fdcfac7fb137c77d1ee021b5cd3ea745377c9ab3e54c0c2276ead040b1ad"}
Oct 30 19:25:04.128995 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:04.128965    4524 scope.go:122] "RemoveContainer" containerID="40917d91e256b667a2c056cdaccf1104ad7080e744a1494302578903c66fe68a"
Oct 30 19:25:04.141304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:04.141275    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp"]
Oct 30 19:25:04.145985 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:04.145960    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-250fb057-7598f9b64-d96bp"]
Oct 30 19:25:04.155186 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:04.155141    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl"]
Oct 30 19:25:04.159754 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:04.159734    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-250fb057-9b44cc67-742xl"]
Oct 30 19:25:04.461931 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:04.461843    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3fc8e7ff-e613-4074-825d-8b9da7356181" path="/var/lib/kubelet/pods/3fc8e7ff-e613-4074-825d-8b9da7356181/volumes"
Oct 30 19:25:04.462336 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:04.462319    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4f6a5962-7e97-4b7d-af82-aa05f7d0785f" path="/var/lib/kubelet/pods/4f6a5962-7e97-4b7d-af82-aa05f7d0785f/volumes"
Oct 30 19:25:05.276915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:05.276871    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt"]
Oct 30 19:25:05.292109 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:05.292071    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk"]
Oct 30 19:25:05.937027 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:05.936465    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"]
Oct 30 19:25:05.937608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:05.937579    4524 cpu_manager.go:414] "containerMap: removing container" podUID="3fc8e7ff-e613-4074-825d-8b9da7356181" containerName="payload-processing"
Oct 30 19:25:05.937608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:05.937607    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3fc8e7ff-e613-4074-825d-8b9da7356181" containerName="payload-processing"
Oct 30 19:25:05.937818 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:05.937670    4524 cpu_manager.go:414] "containerMap: removing container" podUID="4f6a5962-7e97-4b7d-af82-aa05f7d0785f" containerName="payload-pre-processing"
Oct 30 19:25:05.937818 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:05.937679    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4f6a5962-7e97-4b7d-af82-aa05f7d0785f" containerName="payload-pre-processing"
Oct 30 19:25:05.937925 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:05.937842    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="4f6a5962-7e97-4b7d-af82-aa05f7d0785f" containerName="payload-pre-processing"
Oct 30 19:25:05.937925 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:05.937862    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="3fc8e7ff-e613-4074-825d-8b9da7356181" containerName="payload-processing"
Oct 30 19:25:05.944103 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:05.944076    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:05.956322 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:05.956301    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-bce71f\"/\"distinct-2-w6-bce71f-kserve-self-signed-certs\""
Oct 30 19:25:05.961125 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:05.961101    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"]
Oct 30 19:25:05.994422 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:05.994388    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-home\") pod \"distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:05.994628 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:05.994495    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-dshm\") pod \"distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:05.994628 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:05.994545    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59e55265-d446-483c-992c-12134f8ec451-tls-certs\") pod \"distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:05.994628 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:05.994567    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-model-cache\") pod \"distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:05.994829 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:05.994653    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsrxv\" (UniqueName: \"kubernetes.io/projected/59e55265-d446-483c-992c-12134f8ec451-kube-api-access-tsrxv\") pod \"distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:05.994829 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:05.994736    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-tmp-dir\") pod \"distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:06.112207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:06.112146    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59e55265-d446-483c-992c-12134f8ec451-tls-certs\") pod \"distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:06.112382 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:06.112223    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-model-cache\") pod \"distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:06.112382 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:06.112358    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsrxv\" (UniqueName: \"kubernetes.io/projected/59e55265-d446-483c-992c-12134f8ec451-kube-api-access-tsrxv\") pod \"distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:06.112514 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:06.112487    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-tmp-dir\") pod \"distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:06.112618 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:06.112579    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-home\") pod \"distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:06.115204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:06.112767    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-dshm\") pod \"distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:06.115204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:06.113549    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-home\") pod \"distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:06.126195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:06.122570    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-model-cache\") pod \"distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:06.126195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:06.122825    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-tmp-dir\") pod \"distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:06.133016 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:06.131768    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59e55265-d446-483c-992c-12134f8ec451-tls-certs\") pod \"distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:06.170200 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:06.167848    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-dshm\") pod \"distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:06.189720 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:06.178424    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsrxv\" (UniqueName: \"kubernetes.io/projected/59e55265-d446-483c-992c-12134f8ec451-kube-api-access-tsrxv\") pod \"distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") " pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:06.271847 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:06.268484    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:06.623064 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:06.623031    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"]
Oct 30 19:25:06.623873 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:25:06.623847    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59e55265_d446_483c_992c_12134f8ec451.slice/crio-527914c16e5358d06d990f3e91f6209c2d628ddf39b64021e514aaf2d3a7d1a8 WatchSource:0}: Error finding container 527914c16e5358d06d990f3e91f6209c2d628ddf39b64021e514aaf2d3a7d1a8: Status 404 returned error can't find the container with id 527914c16e5358d06d990f3e91f6209c2d628ddf39b64021e514aaf2d3a7d1a8
Oct 30 19:25:06.665513 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:06.665488    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"]
Oct 30 19:25:06.665722 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:06.665698    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q" podUID="5e21b01f-1165-4487-b297-18d11d747650" containerName="maas-api" containerID="cri-o://c0b406bb7da0c41089b7dc06d25e6848809784229b4b7a4ada36bd658d7e19b8" gracePeriod=30
Oct 30 19:25:06.943851 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:06.943761    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"
Oct 30 19:25:06.962612 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:06.962582    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"
Oct 30 19:25:07.141195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.138092    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e21b01f-1165-4487-b297-18d11d747650-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e21b01f-1165-4487-b297-18d11d747650-openshift-service-ca\") pod \"5e21b01f-1165-4487-b297-18d11d747650\" (UID: \"5e21b01f-1165-4487-b297-18d11d747650\") "
Oct 30 19:25:07.141195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.138308    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5e21b01f-1165-4487-b297-18d11d747650-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e21b01f-1165-4487-b297-18d11d747650-maas-api-tls\") pod \"5e21b01f-1165-4487-b297-18d11d747650\" (UID: \"5e21b01f-1165-4487-b297-18d11d747650\") "
Oct 30 19:25:07.141195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.138369    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e21b01f-1165-4487-b297-18d11d747650-kube-api-access-gkqp9\" (UniqueName: \"kubernetes.io/projected/5e21b01f-1165-4487-b297-18d11d747650-kube-api-access-gkqp9\") pod \"5e21b01f-1165-4487-b297-18d11d747650\" (UID: \"5e21b01f-1165-4487-b297-18d11d747650\") "
Oct 30 19:25:07.141195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.138402    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5e21b01f-1165-4487-b297-18d11d747650-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e21b01f-1165-4487-b297-18d11d747650-metrics-certs\") pod \"5e21b01f-1165-4487-b297-18d11d747650\" (UID: \"5e21b01f-1165-4487-b297-18d11d747650\") "
Oct 30 19:25:07.141195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.139963    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e21b01f-1165-4487-b297-18d11d747650-openshift-service-ca" pod "5e21b01f-1165-4487-b297-18d11d747650" (UID: "5e21b01f-1165-4487-b297-18d11d747650"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:25:07.269253 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.262981    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e21b01f-1165-4487-b297-18d11d747650-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:07.362394 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.362356    4524 generic.go:356] "Generic (PLEG): container finished" podID="5e21b01f-1165-4487-b297-18d11d747650" containerID="c0b406bb7da0c41089b7dc06d25e6848809784229b4b7a4ada36bd658d7e19b8" exitCode=0
Oct 30 19:25:07.362669 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.362639    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q" event={"ID":"5e21b01f-1165-4487-b297-18d11d747650","Type":"ContainerDied","Data":"c0b406bb7da0c41089b7dc06d25e6848809784229b4b7a4ada36bd658d7e19b8"}
Oct 30 19:25:07.362833 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.362815    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q" event={"ID":"5e21b01f-1165-4487-b297-18d11d747650","Type":"ContainerDied","Data":"8481d5eb6fe247fdedbfadb2aeb844aff8c3ed5ca83c009439d2f9a5a15ced44"}
Oct 30 19:25:07.362964 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.362949    4524 scope.go:122] "RemoveContainer" containerID="c0b406bb7da0c41089b7dc06d25e6848809784229b4b7a4ada36bd658d7e19b8"
Oct 30 19:25:07.363241 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.363223    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"
Oct 30 19:25:07.374203 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.372361    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8" event={"ID":"59e55265-d446-483c-992c-12134f8ec451","Type":"ContainerStarted","Data":"da1de6ac3906f541933e274167ba7c78178d56fe9e84419544dfdb7365aecb89"}
Oct 30 19:25:07.374203 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.372428    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8" event={"ID":"59e55265-d446-483c-992c-12134f8ec451","Type":"ContainerStarted","Data":"527914c16e5358d06d990f3e91f6209c2d628ddf39b64021e514aaf2d3a7d1a8"}
Oct 30 19:25:07.439208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.430065    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8" podStartSLOduration=2.430043226 podStartE2EDuration="2.430043226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:07.414252382 +0000 UTC m=+1139.155243461" watchObservedRunningTime="2026-10-30 19:25:07.430043226 +0000 UTC m=+1139.171034306"
Oct 30 19:25:07.472828 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.472773    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e21b01f-1165-4487-b297-18d11d747650-kube-api-access-gkqp9" pod "5e21b01f-1165-4487-b297-18d11d747650" (UID: "5e21b01f-1165-4487-b297-18d11d747650"). InnerVolumeSpecName "kube-api-access-gkqp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:25:07.488238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.486387    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e21b01f-1165-4487-b297-18d11d747650-maas-api-tls" pod "5e21b01f-1165-4487-b297-18d11d747650" (UID: "5e21b01f-1165-4487-b297-18d11d747650"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:25:07.490048 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.489380    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e21b01f-1165-4487-b297-18d11d747650-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:07.490048 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.489417    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkqp9\" (UniqueName: \"kubernetes.io/projected/5e21b01f-1165-4487-b297-18d11d747650-kube-api-access-gkqp9\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:07.635798 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.634299    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e21b01f-1165-4487-b297-18d11d747650-metrics-certs" pod "5e21b01f-1165-4487-b297-18d11d747650" (UID: "5e21b01f-1165-4487-b297-18d11d747650"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:25:07.740721 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.716521    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e21b01f-1165-4487-b297-18d11d747650-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:07.745201 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.742393    4524 scope.go:122] "RemoveContainer" containerID="c0b406bb7da0c41089b7dc06d25e6848809784229b4b7a4ada36bd658d7e19b8"
Oct 30 19:25:07.745201 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:25:07.742883    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0b406bb7da0c41089b7dc06d25e6848809784229b4b7a4ada36bd658d7e19b8\": container with ID starting with c0b406bb7da0c41089b7dc06d25e6848809784229b4b7a4ada36bd658d7e19b8 not found: ID does not exist" containerID="c0b406bb7da0c41089b7dc06d25e6848809784229b4b7a4ada36bd658d7e19b8"
Oct 30 19:25:07.745201 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.742921    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0b406bb7da0c41089b7dc06d25e6848809784229b4b7a4ada36bd658d7e19b8"} err="failed to get container status \"c0b406bb7da0c41089b7dc06d25e6848809784229b4b7a4ada36bd658d7e19b8\": rpc error: code = NotFound desc = could not find container \"c0b406bb7da0c41089b7dc06d25e6848809784229b4b7a4ada36bd658d7e19b8\": container with ID starting with c0b406bb7da0c41089b7dc06d25e6848809784229b4b7a4ada36bd658d7e19b8 not found: ID does not exist"
Oct 30 19:25:07.827204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.821031    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"]
Oct 30 19:25:07.860510 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:07.860419    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-250fb057-6d875896d5-q4q5q"]
Oct 30 19:25:08.281453 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:08.273264    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:08.635028 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:08.633443    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e21b01f-1165-4487-b297-18d11d747650" path="/var/lib/kubelet/pods/5e21b01f-1165-4487-b297-18d11d747650/volumes"
Oct 30 19:25:10.277526 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:10.277467    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt" podUID="e40817fb-5abe-4866-9bdf-92dd62c279a5" containerName="payload-processing" containerID="cri-o://0db07c730973917f4c312d9c4bc311a4ca23c588cf704868f71e1c409e74ba90" gracePeriod=25
Oct 30 19:25:10.293159 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:10.293110    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk" podUID="44a66c02-a64d-4188-9cb0-db44dcb6db24" containerName="payload-pre-processing" containerID="cri-o://61548b7fd8a2aef82316744affa046981c92cb3d572859e6a7f32d19a45b0d7c" gracePeriod=25
Oct 30 19:25:10.565699 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:10.565599    4524 generic.go:356] "Generic (PLEG): container finished" podID="e40817fb-5abe-4866-9bdf-92dd62c279a5" containerID="0db07c730973917f4c312d9c4bc311a4ca23c588cf704868f71e1c409e74ba90" exitCode=0
Oct 30 19:25:10.565968 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:10.565888    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt" event={"ID":"e40817fb-5abe-4866-9bdf-92dd62c279a5","Type":"ContainerDied","Data":"0db07c730973917f4c312d9c4bc311a4ca23c588cf704868f71e1c409e74ba90"}
Oct 30 19:25:10.568110 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:10.568042    4524 generic.go:356] "Generic (PLEG): container finished" podID="44a66c02-a64d-4188-9cb0-db44dcb6db24" containerID="61548b7fd8a2aef82316744affa046981c92cb3d572859e6a7f32d19a45b0d7c" exitCode=0
Oct 30 19:25:10.568382 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:10.568074    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk" event={"ID":"44a66c02-a64d-4188-9cb0-db44dcb6db24","Type":"ContainerDied","Data":"61548b7fd8a2aef82316744affa046981c92cb3d572859e6a7f32d19a45b0d7c"}
Oct 30 19:25:10.635917 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:10.635894    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt"
Oct 30 19:25:10.641293 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:10.641268    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk"
Oct 30 19:25:10.825305 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:10.809292    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/44a66c02-a64d-4188-9cb0-db44dcb6db24-kube-api-access-c5rpf\" (UniqueName: \"kubernetes.io/projected/44a66c02-a64d-4188-9cb0-db44dcb6db24-kube-api-access-c5rpf\") pod \"44a66c02-a64d-4188-9cb0-db44dcb6db24\" (UID: \"44a66c02-a64d-4188-9cb0-db44dcb6db24\") "
Oct 30 19:25:10.825305 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:10.809363    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/44a66c02-a64d-4188-9cb0-db44dcb6db24-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44a66c02-a64d-4188-9cb0-db44dcb6db24-plugins-config-volume\") pod \"44a66c02-a64d-4188-9cb0-db44dcb6db24\" (UID: \"44a66c02-a64d-4188-9cb0-db44dcb6db24\") "
Oct 30 19:25:10.825305 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:10.809454    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e40817fb-5abe-4866-9bdf-92dd62c279a5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e40817fb-5abe-4866-9bdf-92dd62c279a5-plugins-config-volume\") pod \"e40817fb-5abe-4866-9bdf-92dd62c279a5\" (UID: \"e40817fb-5abe-4866-9bdf-92dd62c279a5\") "
Oct 30 19:25:10.825305 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:10.809685    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e40817fb-5abe-4866-9bdf-92dd62c279a5-kube-api-access-w45tk\" (UniqueName: \"kubernetes.io/projected/e40817fb-5abe-4866-9bdf-92dd62c279a5-kube-api-access-w45tk\") pod \"e40817fb-5abe-4866-9bdf-92dd62c279a5\" (UID: \"e40817fb-5abe-4866-9bdf-92dd62c279a5\") "
Oct 30 19:25:10.825305 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:10.811360    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e40817fb-5abe-4866-9bdf-92dd62c279a5-plugins-config-volume" pod "e40817fb-5abe-4866-9bdf-92dd62c279a5" (UID: "e40817fb-5abe-4866-9bdf-92dd62c279a5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:25:10.825305 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:10.813008    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44a66c02-a64d-4188-9cb0-db44dcb6db24-plugins-config-volume" pod "44a66c02-a64d-4188-9cb0-db44dcb6db24" (UID: "44a66c02-a64d-4188-9cb0-db44dcb6db24"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:25:10.855394 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:10.855357    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44a66c02-a64d-4188-9cb0-db44dcb6db24-kube-api-access-c5rpf" pod "44a66c02-a64d-4188-9cb0-db44dcb6db24" (UID: "44a66c02-a64d-4188-9cb0-db44dcb6db24"). InnerVolumeSpecName "kube-api-access-c5rpf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:25:10.863546 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:10.861315    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e40817fb-5abe-4866-9bdf-92dd62c279a5-kube-api-access-w45tk" pod "e40817fb-5abe-4866-9bdf-92dd62c279a5" (UID: "e40817fb-5abe-4866-9bdf-92dd62c279a5"). InnerVolumeSpecName "kube-api-access-w45tk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:25:10.915995 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:10.915955    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w45tk\" (UniqueName: \"kubernetes.io/projected/e40817fb-5abe-4866-9bdf-92dd62c279a5-kube-api-access-w45tk\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:10.916207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:10.916001    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5rpf\" (UniqueName: \"kubernetes.io/projected/44a66c02-a64d-4188-9cb0-db44dcb6db24-kube-api-access-c5rpf\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:10.916207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:10.916018    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44a66c02-a64d-4188-9cb0-db44dcb6db24-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:10.916207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:10.916032    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e40817fb-5abe-4866-9bdf-92dd62c279a5-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:11.322639 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:11.322598    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"]
Oct 30 19:25:11.323086 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:11.322839    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn" podUID="aa344907-7495-4eba-811e-e62153b9c8d8" containerName="maas-api" containerID="cri-o://2439d604753e3a1b795d8ad116b8efe7c569d12f294ba8f0df1da5cc2805f145" gracePeriod=30
Oct 30 19:25:11.691273 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:11.678973    4524 generic.go:356] "Generic (PLEG): container finished" podID="aa344907-7495-4eba-811e-e62153b9c8d8" containerID="2439d604753e3a1b795d8ad116b8efe7c569d12f294ba8f0df1da5cc2805f145" exitCode=0
Oct 30 19:25:11.691273 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:11.678988    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn" event={"ID":"aa344907-7495-4eba-811e-e62153b9c8d8","Type":"ContainerDied","Data":"2439d604753e3a1b795d8ad116b8efe7c569d12f294ba8f0df1da5cc2805f145"}
Oct 30 19:25:11.691273 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:11.691055    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk" event={"ID":"44a66c02-a64d-4188-9cb0-db44dcb6db24","Type":"ContainerDied","Data":"7e6aeb560dccb01009db78b9511f8f95807e651a4b47288f69f9842575ffda45"}
Oct 30 19:25:11.691273 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:11.691159    4524 scope.go:122] "RemoveContainer" containerID="61548b7fd8a2aef82316744affa046981c92cb3d572859e6a7f32d19a45b0d7c"
Oct 30 19:25:11.691628 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:11.691409    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk"
Oct 30 19:25:11.736395 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:11.736084    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt" event={"ID":"e40817fb-5abe-4866-9bdf-92dd62c279a5","Type":"ContainerDied","Data":"c55dad6a0e81e1d6636406f82d85f2a25d15f729358088edac4faafd9a0a7988"}
Oct 30 19:25:11.736395 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:11.736244    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt"
Oct 30 19:25:11.785798 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:11.785698    4524 scope.go:122] "RemoveContainer" containerID="0db07c730973917f4c312d9c4bc311a4ca23c588cf704868f71e1c409e74ba90"
Oct 30 19:25:11.848266 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:11.831265    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt"]
Oct 30 19:25:11.853328 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:11.853295    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e308e736-7596fb59d6-95kbt"]
Oct 30 19:25:11.907779 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:11.907725    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk"]
Oct 30 19:25:11.914690 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:11.914657    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e308e736-856845bc79-8vtxk"]
Oct 30 19:25:11.983682 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:11.983651    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"
Oct 30 19:25:12.088198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.081534    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aa344907-7495-4eba-811e-e62153b9c8d8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa344907-7495-4eba-811e-e62153b9c8d8-maas-api-tls\") pod \"aa344907-7495-4eba-811e-e62153b9c8d8\" (UID: \"aa344907-7495-4eba-811e-e62153b9c8d8\") "
Oct 30 19:25:12.088198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.081697    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aa344907-7495-4eba-811e-e62153b9c8d8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa344907-7495-4eba-811e-e62153b9c8d8-metrics-certs\") pod \"aa344907-7495-4eba-811e-e62153b9c8d8\" (UID: \"aa344907-7495-4eba-811e-e62153b9c8d8\") "
Oct 30 19:25:12.088198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.081827    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa344907-7495-4eba-811e-e62153b9c8d8-kube-api-access-m7cpt\" (UniqueName: \"kubernetes.io/projected/aa344907-7495-4eba-811e-e62153b9c8d8-kube-api-access-m7cpt\") pod \"aa344907-7495-4eba-811e-e62153b9c8d8\" (UID: \"aa344907-7495-4eba-811e-e62153b9c8d8\") "
Oct 30 19:25:12.088198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.081910    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa344907-7495-4eba-811e-e62153b9c8d8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa344907-7495-4eba-811e-e62153b9c8d8-openshift-service-ca\") pod \"aa344907-7495-4eba-811e-e62153b9c8d8\" (UID: \"aa344907-7495-4eba-811e-e62153b9c8d8\") "
Oct 30 19:25:12.088198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.086696    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa344907-7495-4eba-811e-e62153b9c8d8-openshift-service-ca" pod "aa344907-7495-4eba-811e-e62153b9c8d8" (UID: "aa344907-7495-4eba-811e-e62153b9c8d8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:25:12.116930 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.116417    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa344907-7495-4eba-811e-e62153b9c8d8-kube-api-access-m7cpt" pod "aa344907-7495-4eba-811e-e62153b9c8d8" (UID: "aa344907-7495-4eba-811e-e62153b9c8d8"). InnerVolumeSpecName "kube-api-access-m7cpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:25:12.117613 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.117585    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa344907-7495-4eba-811e-e62153b9c8d8-maas-api-tls" pod "aa344907-7495-4eba-811e-e62153b9c8d8" (UID: "aa344907-7495-4eba-811e-e62153b9c8d8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:25:12.117720 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.117704    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa344907-7495-4eba-811e-e62153b9c8d8-metrics-certs" pod "aa344907-7495-4eba-811e-e62153b9c8d8" (UID: "aa344907-7495-4eba-811e-e62153b9c8d8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:25:12.184324 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.184287    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa344907-7495-4eba-811e-e62153b9c8d8-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:12.184517 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.184351    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa344907-7495-4eba-811e-e62153b9c8d8-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:12.184517 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.184370    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7cpt\" (UniqueName: \"kubernetes.io/projected/aa344907-7495-4eba-811e-e62153b9c8d8-kube-api-access-m7cpt\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:12.184517 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.184385    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa344907-7495-4eba-811e-e62153b9c8d8-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:12.456340 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.450945    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="44a66c02-a64d-4188-9cb0-db44dcb6db24" path="/var/lib/kubelet/pods/44a66c02-a64d-4188-9cb0-db44dcb6db24/volumes"
Oct 30 19:25:12.458939 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.458903    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e40817fb-5abe-4866-9bdf-92dd62c279a5" path="/var/lib/kubelet/pods/e40817fb-5abe-4866-9bdf-92dd62c279a5/volumes"
Oct 30 19:25:12.710386 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.710355    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5"]
Oct 30 19:25:12.728162 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.728123    4524 cpu_manager.go:414] "containerMap: removing container" podUID="e40817fb-5abe-4866-9bdf-92dd62c279a5" containerName="payload-processing"
Oct 30 19:25:12.761251 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.761212    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e40817fb-5abe-4866-9bdf-92dd62c279a5" containerName="payload-processing"
Oct 30 19:25:12.762374 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.762348    4524 cpu_manager.go:414] "containerMap: removing container" podUID="5e21b01f-1165-4487-b297-18d11d747650" containerName="maas-api"
Oct 30 19:25:12.762517 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.762504    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e21b01f-1165-4487-b297-18d11d747650" containerName="maas-api"
Oct 30 19:25:12.762617 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.762605    4524 cpu_manager.go:414] "containerMap: removing container" podUID="44a66c02-a64d-4188-9cb0-db44dcb6db24" containerName="payload-pre-processing"
Oct 30 19:25:12.762698 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.762686    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="44a66c02-a64d-4188-9cb0-db44dcb6db24" containerName="payload-pre-processing"
Oct 30 19:25:12.762793 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.762784    4524 cpu_manager.go:414] "containerMap: removing container" podUID="aa344907-7495-4eba-811e-e62153b9c8d8" containerName="maas-api"
Oct 30 19:25:12.762874 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.762864    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa344907-7495-4eba-811e-e62153b9c8d8" containerName="maas-api"
Oct 30 19:25:12.776498 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.776465    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="e40817fb-5abe-4866-9bdf-92dd62c279a5" containerName="payload-processing"
Oct 30 19:25:12.776679 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.776666    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e21b01f-1165-4487-b297-18d11d747650" containerName="maas-api"
Oct 30 19:25:12.776767 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.776757    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="44a66c02-a64d-4188-9cb0-db44dcb6db24" containerName="payload-pre-processing"
Oct 30 19:25:12.776847 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.776837    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa344907-7495-4eba-811e-e62153b9c8d8" containerName="maas-api"
Oct 30 19:25:12.879190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.874467    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5"]
Oct 30 19:25:12.879190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.874599    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5"
Oct 30 19:25:12.888668 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.885109    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-c5ed0670\""
Oct 30 19:25:12.888668 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.887283    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-c5ed0670-dockercfg-8cwn4\""
Oct 30 19:25:12.904479 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.904441    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j"]
Oct 30 19:25:12.931529 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.928825    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j"
Oct 30 19:25:12.996832 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.996184    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j\" (UID: \"fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j"
Oct 30 19:25:12.996832 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.996269    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q589r\" (UniqueName: \"kubernetes.io/projected/e94b5346-79f3-494c-bfba-a72938a84194-kube-api-access-q589r\") pod \"payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5\" (UID: \"e94b5346-79f3-494c-bfba-a72938a84194\") " pod="openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5"
Oct 30 19:25:12.996832 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.996371    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkftp\" (UniqueName: \"kubernetes.io/projected/fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab-kube-api-access-gkftp\") pod \"payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j\" (UID: \"fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j"
Oct 30 19:25:12.996832 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.996517    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e94b5346-79f3-494c-bfba-a72938a84194-plugins-config-volume\") pod \"payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5\" (UID: \"e94b5346-79f3-494c-bfba-a72938a84194\") " pod="openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5"
Oct 30 19:25:13.006220 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:12.997894    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"
Oct 30 19:25:13.016212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:13.011855    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn" event={"ID":"aa344907-7495-4eba-811e-e62153b9c8d8","Type":"ContainerDied","Data":"40dea5f76f476d5b56f25d7ae0e3078d70cb83ac438cd6629f63965cef371861"}
Oct 30 19:25:13.016212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:13.011974    4524 scope.go:122] "RemoveContainer" containerID="2439d604753e3a1b795d8ad116b8efe7c569d12f294ba8f0df1da5cc2805f145"
Oct 30 19:25:13.055416 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:13.055313    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j"]
Oct 30 19:25:13.084816 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:13.084785    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"]
Oct 30 19:25:13.092981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:13.092951    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e308e736-58444d988d-zx6tn"]
Oct 30 19:25:13.098678 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:13.098515    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j\" (UID: \"fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j"
Oct 30 19:25:13.098678 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:13.098581    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q589r\" (UniqueName: \"kubernetes.io/projected/e94b5346-79f3-494c-bfba-a72938a84194-kube-api-access-q589r\") pod \"payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5\" (UID: \"e94b5346-79f3-494c-bfba-a72938a84194\") " pod="openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5"
Oct 30 19:25:13.098678 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:13.098664    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkftp\" (UniqueName: \"kubernetes.io/projected/fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab-kube-api-access-gkftp\") pod \"payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j\" (UID: \"fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j"
Oct 30 19:25:13.098916 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:13.098765    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e94b5346-79f3-494c-bfba-a72938a84194-plugins-config-volume\") pod \"payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5\" (UID: \"e94b5346-79f3-494c-bfba-a72938a84194\") " pod="openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5"
Oct 30 19:25:13.099557 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:13.099530    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j\" (UID: \"fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j"
Oct 30 19:25:13.100253 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:13.100226    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e94b5346-79f3-494c-bfba-a72938a84194-plugins-config-volume\") pod \"payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5\" (UID: \"e94b5346-79f3-494c-bfba-a72938a84194\") " pod="openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5"
Oct 30 19:25:13.110630 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:13.110603    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkftp\" (UniqueName: \"kubernetes.io/projected/fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab-kube-api-access-gkftp\") pod \"payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j\" (UID: \"fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j"
Oct 30 19:25:13.110714 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:13.110604    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q589r\" (UniqueName: \"kubernetes.io/projected/e94b5346-79f3-494c-bfba-a72938a84194-kube-api-access-q589r\") pod \"payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5\" (UID: \"e94b5346-79f3-494c-bfba-a72938a84194\") " pod="openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5"
Oct 30 19:25:13.239624 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:13.239362    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5"
Oct 30 19:25:13.297810 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:13.297779    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j"
Oct 30 19:25:13.760484 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:13.760447    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5"]
Oct 30 19:25:13.873287 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:25:13.873262    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa4b7e4b_b8fa_4c63_b119_4516cc9c59ab.slice/crio-3da6da4a9d0a31e38dba6f62ae09fb606fb0776921a87e2ddefaa73813f9c075 WatchSource:0}: Error finding container 3da6da4a9d0a31e38dba6f62ae09fb606fb0776921a87e2ddefaa73813f9c075: Status 404 returned error can't find the container with id 3da6da4a9d0a31e38dba6f62ae09fb606fb0776921a87e2ddefaa73813f9c075
Oct 30 19:25:13.877262 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:13.877147    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j"]
Oct 30 19:25:14.069573 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:14.069532    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j" event={"ID":"fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab","Type":"ContainerStarted","Data":"bbdbc8e3e6aafd6652df4428b39f53b6257596193f0a44a5c51bb4650d0c3503"}
Oct 30 19:25:14.069747 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:14.069601    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j" event={"ID":"fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab","Type":"ContainerStarted","Data":"3da6da4a9d0a31e38dba6f62ae09fb606fb0776921a87e2ddefaa73813f9c075"}
Oct 30 19:25:14.069747 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:14.069633    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j"
Oct 30 19:25:14.072404 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:14.072366    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5" event={"ID":"e94b5346-79f3-494c-bfba-a72938a84194","Type":"ContainerStarted","Data":"5a85b9a87b60418861d8aafbe62f422b150187f7734996562b8502cb01d3720f"}
Oct 30 19:25:14.072540 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:14.072460    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5" event={"ID":"e94b5346-79f3-494c-bfba-a72938a84194","Type":"ContainerStarted","Data":"89f66d97abf5d0681cb76dd067df5867e987b5c73ef33cf85d6d8b4aee25a9df"}
Oct 30 19:25:14.072540 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:14.072495    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5"
Oct 30 19:25:14.092798 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:14.092739    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j" podStartSLOduration=2.09272085 podStartE2EDuration="2.09272085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:14.088375031 +0000 UTC m=+1145.829366110" watchObservedRunningTime="2026-10-30 19:25:14.09272085 +0000 UTC m=+1145.833711931"
Oct 30 19:25:14.110907 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:14.110838    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5" podStartSLOduration=2.110826358 podStartE2EDuration="2.110826358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:14.106736493 +0000 UTC m=+1145.847727571" watchObservedRunningTime="2026-10-30 19:25:14.110826358 +0000 UTC m=+1145.851817436"
Oct 30 19:25:14.394994 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:14.394946    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa344907-7495-4eba-811e-e62153b9c8d8" path="/var/lib/kubelet/pods/aa344907-7495-4eba-811e-e62153b9c8d8/volumes"
Oct 30 19:25:15.628436 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:25:15.628393    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:25:16.275270 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:16.275238    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:16.306697 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:16.306614    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:17.130771 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:17.130744    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:18.866705 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:18.866666    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"]
Oct 30 19:25:18.874095 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:18.874067    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"
Oct 30 19:25:18.893596 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:18.893565    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"]
Oct 30 19:25:18.895468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:18.895434    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-c5ed0670-serving-cert\""
Oct 30 19:25:19.000299 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:19.000256    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36cee36f-9e21-4e28-9091-95f051e3a94f-maas-api-tls\") pod \"maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4\" (UID: \"36cee36f-9e21-4e28-9091-95f051e3a94f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"
Oct 30 19:25:19.015200 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:19.013788    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsdnf\" (UniqueName: \"kubernetes.io/projected/36cee36f-9e21-4e28-9091-95f051e3a94f-kube-api-access-dsdnf\") pod \"maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4\" (UID: \"36cee36f-9e21-4e28-9091-95f051e3a94f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"
Oct 30 19:25:19.015200 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:19.013956    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36cee36f-9e21-4e28-9091-95f051e3a94f-metrics-certs\") pod \"maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4\" (UID: \"36cee36f-9e21-4e28-9091-95f051e3a94f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"
Oct 30 19:25:19.015200 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:19.014012    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36cee36f-9e21-4e28-9091-95f051e3a94f-openshift-service-ca\") pod \"maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4\" (UID: \"36cee36f-9e21-4e28-9091-95f051e3a94f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"
Oct 30 19:25:19.126303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:19.115687    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36cee36f-9e21-4e28-9091-95f051e3a94f-maas-api-tls\") pod \"maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4\" (UID: \"36cee36f-9e21-4e28-9091-95f051e3a94f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"
Oct 30 19:25:19.126303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:19.115779    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsdnf\" (UniqueName: \"kubernetes.io/projected/36cee36f-9e21-4e28-9091-95f051e3a94f-kube-api-access-dsdnf\") pod \"maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4\" (UID: \"36cee36f-9e21-4e28-9091-95f051e3a94f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"
Oct 30 19:25:19.126303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:19.115894    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36cee36f-9e21-4e28-9091-95f051e3a94f-metrics-certs\") pod \"maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4\" (UID: \"36cee36f-9e21-4e28-9091-95f051e3a94f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"
Oct 30 19:25:19.126303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:19.115942    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36cee36f-9e21-4e28-9091-95f051e3a94f-openshift-service-ca\") pod \"maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4\" (UID: \"36cee36f-9e21-4e28-9091-95f051e3a94f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"
Oct 30 19:25:19.126303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:19.119693    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36cee36f-9e21-4e28-9091-95f051e3a94f-openshift-service-ca\") pod \"maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4\" (UID: \"36cee36f-9e21-4e28-9091-95f051e3a94f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"
Oct 30 19:25:19.126303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:19.125055    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36cee36f-9e21-4e28-9091-95f051e3a94f-metrics-certs\") pod \"maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4\" (UID: \"36cee36f-9e21-4e28-9091-95f051e3a94f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"
Oct 30 19:25:19.126303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:19.125557    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36cee36f-9e21-4e28-9091-95f051e3a94f-maas-api-tls\") pod \"maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4\" (UID: \"36cee36f-9e21-4e28-9091-95f051e3a94f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"
Oct 30 19:25:19.141244 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:19.138467    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsdnf\" (UniqueName: \"kubernetes.io/projected/36cee36f-9e21-4e28-9091-95f051e3a94f-kube-api-access-dsdnf\") pod \"maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4\" (UID: \"36cee36f-9e21-4e28-9091-95f051e3a94f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"
Oct 30 19:25:19.216340 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:19.216311    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"
Oct 30 19:25:19.510567 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:19.510534    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"]
Oct 30 19:25:19.512110 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:25:19.512050    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36cee36f_9e21_4e28_9091_95f051e3a94f.slice/crio-eee55feaebbd9e4b79831c61a756775f2293d5eeb3955afd236fc8835fbf3b30 WatchSource:0}: Error finding container eee55feaebbd9e4b79831c61a756775f2293d5eeb3955afd236fc8835fbf3b30: Status 404 returned error can't find the container with id eee55feaebbd9e4b79831c61a756775f2293d5eeb3955afd236fc8835fbf3b30
Oct 30 19:25:20.218720 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.218627    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4" event={"ID":"36cee36f-9e21-4e28-9091-95f051e3a94f","Type":"ContainerStarted","Data":"1a0c7cbca8c0877881fae334b489870784e9dd5425b9ceca50be0632f71bb65f"}
Oct 30 19:25:20.218720 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.218689    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4" event={"ID":"36cee36f-9e21-4e28-9091-95f051e3a94f","Type":"ContainerStarted","Data":"eee55feaebbd9e4b79831c61a756775f2293d5eeb3955afd236fc8835fbf3b30"}
Oct 30 19:25:20.219738 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.219711    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"
Oct 30 19:25:20.227536 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.227512    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"]
Oct 30 19:25:20.232482 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.232464    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.263205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.262917    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"]
Oct 30 19:25:20.263581 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.263529    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4" podStartSLOduration=1.8120056839999998 podStartE2EDuration="2.26351235s" totalImagesPullingTime="451.506666ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:20.254603914 +0000 UTC m=+1151.995594993" watchObservedRunningTime="2026-10-30 19:25:20.26351235 +0000 UTC m=+1152.004503430"
Oct 30 19:25:20.270135 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.270111    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-workload-certs\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.270843 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.270819    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-envoy\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.270988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.270968    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-podinfo\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.271214 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.271152    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-data\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.271321 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.271231    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-credential-socket\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.271321 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.271278    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-workload-socket\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.273276 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.273249    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-token\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.273388 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.273370    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r56r9\" (UniqueName: \"kubernetes.io/projected/22c6a6f8-5d6b-4607-85b1-881a6a08f361-kube-api-access-r56r9\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.273532 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.273515    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istiod-ca-cert\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.399064 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.387162    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-token\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.399064 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.387271    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r56r9\" (UniqueName: \"kubernetes.io/projected/22c6a6f8-5d6b-4607-85b1-881a6a08f361-kube-api-access-r56r9\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.399064 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.387362    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istiod-ca-cert\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.399064 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.387475    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-workload-certs\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.399064 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.387528    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-envoy\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.399064 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.387585    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-podinfo\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.399064 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.387652    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-data\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.399064 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.387687    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-credential-socket\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.399064 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.396904    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-workload-socket\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.399064 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.397513    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-workload-socket\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.438195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.437977    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istiod-ca-cert\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.443199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.439812    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-data\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.443199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.440438    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-podinfo\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.443199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.440675    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-workload-certs\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.443199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.440853    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-credential-socket\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.488362 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.488259    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-envoy\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.491398 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.491369    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-token\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.492738 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.492709    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r56r9\" (UniqueName: \"kubernetes.io/projected/22c6a6f8-5d6b-4607-85b1-881a6a08f361-kube-api-access-r56r9\") pod \"e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") " pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.572946 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.572915    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:20.965391 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:25:20.963539    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22c6a6f8_5d6b_4607_85b1_881a6a08f361.slice/crio-5b1693faebe168c4562aa2471de3d387c437e737d39b105ac29b6bd483c6549d WatchSource:0}: Error finding container 5b1693faebe168c4562aa2471de3d387c437e737d39b105ac29b6bd483c6549d: Status 404 returned error can't find the container with id 5b1693faebe168c4562aa2471de3d387c437e737d39b105ac29b6bd483c6549d
Oct 30 19:25:20.981364 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.980483    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:25:20.981364 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.980582    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:25:20.981364 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.980629    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:25:20.982100 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:20.981625    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"]
Oct 30 19:25:21.330192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:21.328527    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9" event={"ID":"22c6a6f8-5d6b-4607-85b1-881a6a08f361","Type":"ContainerStarted","Data":"5b1693faebe168c4562aa2471de3d387c437e737d39b105ac29b6bd483c6549d"}
Oct 30 19:25:22.592196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:22.574837    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9" event={"ID":"22c6a6f8-5d6b-4607-85b1-881a6a08f361","Type":"ContainerStarted","Data":"eeba74b9d84738467ad94bccb4b77b32665329861b0a6d3ec9001f84148eae22"}
Oct 30 19:25:22.592196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:22.582118    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:22.622644 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:22.618577    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9" podStartSLOduration=2.618555861 podStartE2EDuration="2.618555861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:22.605189523 +0000 UTC m=+1154.346180606" watchObservedRunningTime="2026-10-30 19:25:22.618555861 +0000 UTC m=+1154.359546941"
Oct 30 19:25:22.639345 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:22.638445    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:25:22.639345 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:22.638537    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9" podUID="22c6a6f8-5d6b-4607-85b1-881a6a08f361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 19:25:23.581309 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:23.580838    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:25:23.581309 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:23.580895    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9" podUID="22c6a6f8-5d6b-4607-85b1-881a6a08f361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 19:25:24.653224 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:24.648658    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:25:24.653841 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:24.653801    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9" podUID="22c6a6f8-5d6b-4607-85b1-881a6a08f361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 19:25:25.127643 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:25.127608    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j"
Oct 30 19:25:25.130026 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:25.129996    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc"]
Oct 30 19:25:25.259106 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:25.172193    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8" podUID="59e55265-d446-483c-992c-12134f8ec451" containerName="main" probeResult="failure" output="Get \"https://10.133.0.101:8000/ready\": context deadline exceeded"
Oct 30 19:25:25.259331 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:25.246283    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz" podUID="e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:25:25.278438 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:25.277702    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5"
Oct 30 19:25:25.608194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:25.604544    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:25:25.608194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:25.604621    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9" podUID="22c6a6f8-5d6b-4607-85b1-881a6a08f361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 19:25:25.849012 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:25.848966    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j" podUID="24fbbf1e-3438-4436-b4cd-9890fe28676a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.98:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:25:26.264273 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:26.262497    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8" podUID="59e55265-d446-483c-992c-12134f8ec451" containerName="main" probeResult="failure" output="Get \"https://10.133.0.101:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:25:26.284563 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:26.269676    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:26.284563 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:26.282152    4524 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j" podUID="24fbbf1e-3438-4436-b4cd-9890fe28676a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.98:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:25:26.422330 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:26.422269    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz" podUID="e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:25:26.585258 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:26.580984    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:25:26.585258 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:26.581052    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9" podUID="22c6a6f8-5d6b-4607-85b1-881a6a08f361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 19:25:26.930254 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:26.909307    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:25:26.930254 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:26.909371    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:25:26.930254 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:26.909403    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:25:27.612193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:27.608566    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:25:27.612193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:27.608682    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9" podUID="22c6a6f8-5d6b-4607-85b1-881a6a08f361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 19:25:27.743196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:27.739408    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"
Oct 30 19:25:28.589531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:28.589494    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:25:28.590114 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:28.590080    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9" podUID="22c6a6f8-5d6b-4607-85b1-881a6a08f361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 19:25:29.586206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:29.575550    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:25:29.586206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:29.575616    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9" podUID="22c6a6f8-5d6b-4607-85b1-881a6a08f361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 19:25:29.757635 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:29.757590    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"]
Oct 30 19:25:29.758036 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:29.757784    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p" podUID="445c0240-a758-4f58-80ac-9c99afc0a2d6" containerName="maas-api" containerID="cri-o://6305cfba466c3695801614872480b88a157cae8714d833d29b02bda0860159f1" gracePeriod=30
Oct 30 19:25:30.287715 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:30.287658    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc" podUID="c57d766d-cf24-4997-aeca-810860a18ca1" containerName="payload-processing" containerID="cri-o://a714d925a0c9e5de58f8ecaa73164a76a8ffa78df36782ccbfe5cff0c60f4d5e" gracePeriod=25
Oct 30 19:25:30.498846 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:30.498809    4524 generic.go:356] "Generic (PLEG): container finished" podID="445c0240-a758-4f58-80ac-9c99afc0a2d6" containerID="6305cfba466c3695801614872480b88a157cae8714d833d29b02bda0860159f1" exitCode=0
Oct 30 19:25:30.499369 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:30.499342    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p" event={"ID":"445c0240-a758-4f58-80ac-9c99afc0a2d6","Type":"ContainerDied","Data":"6305cfba466c3695801614872480b88a157cae8714d833d29b02bda0860159f1"}
Oct 30 19:25:30.575758 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:30.575496    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:25:30.575758 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:30.575560    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9" podUID="22c6a6f8-5d6b-4607-85b1-881a6a08f361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 19:25:30.623832 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:30.623803    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"
Oct 30 19:25:30.677708 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:30.673660    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/445c0240-a758-4f58-80ac-9c99afc0a2d6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/445c0240-a758-4f58-80ac-9c99afc0a2d6-metrics-certs\") pod \"445c0240-a758-4f58-80ac-9c99afc0a2d6\" (UID: \"445c0240-a758-4f58-80ac-9c99afc0a2d6\") "
Oct 30 19:25:30.677708 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:30.673707    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/445c0240-a758-4f58-80ac-9c99afc0a2d6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/445c0240-a758-4f58-80ac-9c99afc0a2d6-openshift-service-ca\") pod \"445c0240-a758-4f58-80ac-9c99afc0a2d6\" (UID: \"445c0240-a758-4f58-80ac-9c99afc0a2d6\") "
Oct 30 19:25:30.677708 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:30.673857    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/445c0240-a758-4f58-80ac-9c99afc0a2d6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/445c0240-a758-4f58-80ac-9c99afc0a2d6-maas-api-tls\") pod \"445c0240-a758-4f58-80ac-9c99afc0a2d6\" (UID: \"445c0240-a758-4f58-80ac-9c99afc0a2d6\") "
Oct 30 19:25:30.677708 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:30.673980    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/445c0240-a758-4f58-80ac-9c99afc0a2d6-kube-api-access-qxvzn\" (UniqueName: \"kubernetes.io/projected/445c0240-a758-4f58-80ac-9c99afc0a2d6-kube-api-access-qxvzn\") pod \"445c0240-a758-4f58-80ac-9c99afc0a2d6\" (UID: \"445c0240-a758-4f58-80ac-9c99afc0a2d6\") "
Oct 30 19:25:30.677708 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:30.675506    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/445c0240-a758-4f58-80ac-9c99afc0a2d6-openshift-service-ca" pod "445c0240-a758-4f58-80ac-9c99afc0a2d6" (UID: "445c0240-a758-4f58-80ac-9c99afc0a2d6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:25:30.684409 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:30.680145    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/445c0240-a758-4f58-80ac-9c99afc0a2d6-maas-api-tls" pod "445c0240-a758-4f58-80ac-9c99afc0a2d6" (UID: "445c0240-a758-4f58-80ac-9c99afc0a2d6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:25:30.684409 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:30.683635    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/445c0240-a758-4f58-80ac-9c99afc0a2d6-kube-api-access-qxvzn" pod "445c0240-a758-4f58-80ac-9c99afc0a2d6" (UID: "445c0240-a758-4f58-80ac-9c99afc0a2d6"). InnerVolumeSpecName "kube-api-access-qxvzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:25:30.686973 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:30.685664    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/445c0240-a758-4f58-80ac-9c99afc0a2d6-metrics-certs" pod "445c0240-a758-4f58-80ac-9c99afc0a2d6" (UID: "445c0240-a758-4f58-80ac-9c99afc0a2d6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:25:30.831407 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:30.831292    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/445c0240-a758-4f58-80ac-9c99afc0a2d6-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:30.831407 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:30.831351    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/445c0240-a758-4f58-80ac-9c99afc0a2d6-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:30.831407 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:30.831369    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/445c0240-a758-4f58-80ac-9c99afc0a2d6-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:30.831407 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:30.831384    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxvzn\" (UniqueName: \"kubernetes.io/projected/445c0240-a758-4f58-80ac-9c99afc0a2d6-kube-api-access-qxvzn\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:30.968425 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:30.968394    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc"
Oct 30 19:25:31.046157 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.045933    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c57d766d-cf24-4997-aeca-810860a18ca1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c57d766d-cf24-4997-aeca-810860a18ca1-plugins-config-volume\") pod \"c57d766d-cf24-4997-aeca-810860a18ca1\" (UID: \"c57d766d-cf24-4997-aeca-810860a18ca1\") "
Oct 30 19:25:31.046157 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.046099    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c57d766d-cf24-4997-aeca-810860a18ca1-kube-api-access-sn9hz\" (UniqueName: \"kubernetes.io/projected/c57d766d-cf24-4997-aeca-810860a18ca1-kube-api-access-sn9hz\") pod \"c57d766d-cf24-4997-aeca-810860a18ca1\" (UID: \"c57d766d-cf24-4997-aeca-810860a18ca1\") "
Oct 30 19:25:31.047139 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.047112    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c57d766d-cf24-4997-aeca-810860a18ca1-plugins-config-volume" pod "c57d766d-cf24-4997-aeca-810860a18ca1" (UID: "c57d766d-cf24-4997-aeca-810860a18ca1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:25:31.053606 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.053576    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c57d766d-cf24-4997-aeca-810860a18ca1-kube-api-access-sn9hz" pod "c57d766d-cf24-4997-aeca-810860a18ca1" (UID: "c57d766d-cf24-4997-aeca-810860a18ca1"). InnerVolumeSpecName "kube-api-access-sn9hz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:25:31.149044 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.148961    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c57d766d-cf24-4997-aeca-810860a18ca1-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:31.149044 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.148995    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sn9hz\" (UniqueName: \"kubernetes.io/projected/c57d766d-cf24-4997-aeca-810860a18ca1-kube-api-access-sn9hz\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:31.528645 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.528540    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p" event={"ID":"445c0240-a758-4f58-80ac-9c99afc0a2d6","Type":"ContainerDied","Data":"5551b80f9f7bedfad33177e086f47e5f54769926e4c150d74e5fab88e2848a98"}
Oct 30 19:25:31.528645 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.528615    4524 scope.go:122] "RemoveContainer" containerID="6305cfba466c3695801614872480b88a157cae8714d833d29b02bda0860159f1"
Oct 30 19:25:31.528872 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.528561    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"
Oct 30 19:25:31.533604 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.533419    4524 generic.go:356] "Generic (PLEG): container finished" podID="c57d766d-cf24-4997-aeca-810860a18ca1" containerID="a714d925a0c9e5de58f8ecaa73164a76a8ffa78df36782ccbfe5cff0c60f4d5e" exitCode=0
Oct 30 19:25:31.533604 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.533456    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc"
Oct 30 19:25:31.533604 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.533527    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc" event={"ID":"c57d766d-cf24-4997-aeca-810860a18ca1","Type":"ContainerDied","Data":"a714d925a0c9e5de58f8ecaa73164a76a8ffa78df36782ccbfe5cff0c60f4d5e"}
Oct 30 19:25:31.533604 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.533563    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc" event={"ID":"c57d766d-cf24-4997-aeca-810860a18ca1","Type":"ContainerDied","Data":"15fdbba332957b0bb677a3461e08d8b67d49df23277b415fd907d091c076ae28"}
Oct 30 19:25:31.548115 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.547276    4524 scope.go:122] "RemoveContainer" containerID="a714d925a0c9e5de58f8ecaa73164a76a8ffa78df36782ccbfe5cff0c60f4d5e"
Oct 30 19:25:31.557343 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.557315    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"]
Oct 30 19:25:31.559063 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.559038    4524 scope.go:122] "RemoveContainer" containerID="a714d925a0c9e5de58f8ecaa73164a76a8ffa78df36782ccbfe5cff0c60f4d5e"
Oct 30 19:25:31.559451 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:25:31.559432    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a714d925a0c9e5de58f8ecaa73164a76a8ffa78df36782ccbfe5cff0c60f4d5e\": container with ID starting with a714d925a0c9e5de58f8ecaa73164a76a8ffa78df36782ccbfe5cff0c60f4d5e not found: ID does not exist" containerID="a714d925a0c9e5de58f8ecaa73164a76a8ffa78df36782ccbfe5cff0c60f4d5e"
Oct 30 19:25:31.559528 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.559467    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a714d925a0c9e5de58f8ecaa73164a76a8ffa78df36782ccbfe5cff0c60f4d5e"} err="failed to get container status \"a714d925a0c9e5de58f8ecaa73164a76a8ffa78df36782ccbfe5cff0c60f4d5e\": rpc error: code = NotFound desc = could not find container \"a714d925a0c9e5de58f8ecaa73164a76a8ffa78df36782ccbfe5cff0c60f4d5e\": container with ID starting with a714d925a0c9e5de58f8ecaa73164a76a8ffa78df36782ccbfe5cff0c60f4d5e not found: ID does not exist"
Oct 30 19:25:31.561977 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.561957    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7c74535f-7668697cbb-z8n2p"]
Oct 30 19:25:31.573287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.573259    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused" start-of-body=
Oct 30 19:25:31.573410 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.573315    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9" podUID="22c6a6f8-5d6b-4607-85b1-881a6a08f361" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.105:15021/healthz/ready\": dial tcp 10.133.0.105:15021: connect: connection refused"
Oct 30 19:25:31.575520 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.575495    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc"]
Oct 30 19:25:31.579407 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:31.579387    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7c74535f-74c84655b6-pzgqc"]
Oct 30 19:25:32.395122 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:32.395084    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="445c0240-a758-4f58-80ac-9c99afc0a2d6" path="/var/lib/kubelet/pods/445c0240-a758-4f58-80ac-9c99afc0a2d6/volumes"
Oct 30 19:25:32.395655 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:32.395530    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c57d766d-cf24-4997-aeca-810860a18ca1" path="/var/lib/kubelet/pods/c57d766d-cf24-4997-aeca-810860a18ca1/volumes"
Oct 30 19:25:32.777213 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:32.775911    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:32.777213 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:32.776886    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:32.783824 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:32.779831    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:25:45.627788 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:25:45.627667    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:25:46.519355 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:46.519310    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5"]
Oct 30 19:25:46.527667 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:46.527633    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j"]
Oct 30 19:25:47.136125 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.135831    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"]
Oct 30 19:25:47.137917 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.137304    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4" podUID="36cee36f-9e21-4e28-9091-95f051e3a94f" containerName="maas-api" containerID="cri-o://1a0c7cbca8c0877881fae334b489870784e9dd5425b9ceca50be0632f71bb65f" gracePeriod=30
Oct 30 19:25:47.420009 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.410141    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk"]
Oct 30 19:25:47.420009 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.411232    4524 cpu_manager.go:414] "containerMap: removing container" podUID="c57d766d-cf24-4997-aeca-810860a18ca1" containerName="payload-processing"
Oct 30 19:25:47.420009 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.411253    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c57d766d-cf24-4997-aeca-810860a18ca1" containerName="payload-processing"
Oct 30 19:25:47.420009 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.411314    4524 cpu_manager.go:414] "containerMap: removing container" podUID="445c0240-a758-4f58-80ac-9c99afc0a2d6" containerName="maas-api"
Oct 30 19:25:47.420009 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.411322    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="445c0240-a758-4f58-80ac-9c99afc0a2d6" containerName="maas-api"
Oct 30 19:25:47.420009 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.411481    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="c57d766d-cf24-4997-aeca-810860a18ca1" containerName="payload-processing"
Oct 30 19:25:47.420009 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.411502    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="445c0240-a758-4f58-80ac-9c99afc0a2d6" containerName="maas-api"
Oct 30 19:25:47.420009 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.416036    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87"]
Oct 30 19:25:47.420009 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.419364    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk"
Oct 30 19:25:47.430220 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.422600    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87"
Oct 30 19:25:47.430220 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.423184    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w0-2405fb-dockercfg-7bpwx\""
Oct 30 19:25:47.430220 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.423519    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w0-2405fb\""
Oct 30 19:25:47.430220 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.424732    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk"]
Oct 30 19:25:47.455035 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.454995    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87"]
Oct 30 19:25:47.497223 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.497158    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgc4p\" (UniqueName: \"kubernetes.io/projected/1c50188a-2ca9-4063-9092-9fdd095582aa-kube-api-access-dgc4p\") pod \"payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87\" (UID: \"1c50188a-2ca9-4063-9092-9fdd095582aa\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87"
Oct 30 19:25:47.497422 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.497399    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c50188a-2ca9-4063-9092-9fdd095582aa-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87\" (UID: \"1c50188a-2ca9-4063-9092-9fdd095582aa\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87"
Oct 30 19:25:47.497614 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.497592    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/533f548c-cf0b-4bd7-9fed-2d9bde66ec95-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk\" (UID: \"533f548c-cf0b-4bd7-9fed-2d9bde66ec95\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk"
Oct 30 19:25:47.497687 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.497643    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnrtx\" (UniqueName: \"kubernetes.io/projected/533f548c-cf0b-4bd7-9fed-2d9bde66ec95-kube-api-access-pnrtx\") pod \"payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk\" (UID: \"533f548c-cf0b-4bd7-9fed-2d9bde66ec95\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk"
Oct 30 19:25:47.611369 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.611320    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c50188a-2ca9-4063-9092-9fdd095582aa-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87\" (UID: \"1c50188a-2ca9-4063-9092-9fdd095582aa\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87"
Oct 30 19:25:47.611842 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.611811    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/533f548c-cf0b-4bd7-9fed-2d9bde66ec95-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk\" (UID: \"533f548c-cf0b-4bd7-9fed-2d9bde66ec95\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk"
Oct 30 19:25:47.612013 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.611997    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnrtx\" (UniqueName: \"kubernetes.io/projected/533f548c-cf0b-4bd7-9fed-2d9bde66ec95-kube-api-access-pnrtx\") pod \"payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk\" (UID: \"533f548c-cf0b-4bd7-9fed-2d9bde66ec95\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk"
Oct 30 19:25:47.612265 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.612243    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgc4p\" (UniqueName: \"kubernetes.io/projected/1c50188a-2ca9-4063-9092-9fdd095582aa-kube-api-access-dgc4p\") pod \"payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87\" (UID: \"1c50188a-2ca9-4063-9092-9fdd095582aa\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87"
Oct 30 19:25:47.613767 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.613739    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c50188a-2ca9-4063-9092-9fdd095582aa-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87\" (UID: \"1c50188a-2ca9-4063-9092-9fdd095582aa\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87"
Oct 30 19:25:47.630743 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.614708    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/533f548c-cf0b-4bd7-9fed-2d9bde66ec95-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk\" (UID: \"533f548c-cf0b-4bd7-9fed-2d9bde66ec95\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk"
Oct 30 19:25:47.657196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.650323    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"
Oct 30 19:25:47.661881 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.661846    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnrtx\" (UniqueName: \"kubernetes.io/projected/533f548c-cf0b-4bd7-9fed-2d9bde66ec95-kube-api-access-pnrtx\") pod \"payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk\" (UID: \"533f548c-cf0b-4bd7-9fed-2d9bde66ec95\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk"
Oct 30 19:25:47.693142 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.693060    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgc4p\" (UniqueName: \"kubernetes.io/projected/1c50188a-2ca9-4063-9092-9fdd095582aa-kube-api-access-dgc4p\") pod \"payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87\" (UID: \"1c50188a-2ca9-4063-9092-9fdd095582aa\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87"
Oct 30 19:25:47.716698 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.716657    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36cee36f-9e21-4e28-9091-95f051e3a94f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36cee36f-9e21-4e28-9091-95f051e3a94f-openshift-service-ca\") pod \"36cee36f-9e21-4e28-9091-95f051e3a94f\" (UID: \"36cee36f-9e21-4e28-9091-95f051e3a94f\") "
Oct 30 19:25:47.757059 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.757030    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36cee36f-9e21-4e28-9091-95f051e3a94f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36cee36f-9e21-4e28-9091-95f051e3a94f-maas-api-tls\") pod \"36cee36f-9e21-4e28-9091-95f051e3a94f\" (UID: \"36cee36f-9e21-4e28-9091-95f051e3a94f\") "
Oct 30 19:25:47.757932 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.757904    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36cee36f-9e21-4e28-9091-95f051e3a94f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/36cee36f-9e21-4e28-9091-95f051e3a94f-metrics-certs\") pod \"36cee36f-9e21-4e28-9091-95f051e3a94f\" (UID: \"36cee36f-9e21-4e28-9091-95f051e3a94f\") "
Oct 30 19:25:47.765326 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.765298    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36cee36f-9e21-4e28-9091-95f051e3a94f-kube-api-access-dsdnf\" (UniqueName: \"kubernetes.io/projected/36cee36f-9e21-4e28-9091-95f051e3a94f-kube-api-access-dsdnf\") pod \"36cee36f-9e21-4e28-9091-95f051e3a94f\" (UID: \"36cee36f-9e21-4e28-9091-95f051e3a94f\") "
Oct 30 19:25:47.785192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.720933    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36cee36f-9e21-4e28-9091-95f051e3a94f-openshift-service-ca" pod "36cee36f-9e21-4e28-9091-95f051e3a94f" (UID: "36cee36f-9e21-4e28-9091-95f051e3a94f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:25:47.785192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.752569    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk"
Oct 30 19:25:47.786485 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.756612    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87"
Oct 30 19:25:47.818243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.812411    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36cee36f-9e21-4e28-9091-95f051e3a94f-kube-api-access-dsdnf" pod "36cee36f-9e21-4e28-9091-95f051e3a94f" (UID: "36cee36f-9e21-4e28-9091-95f051e3a94f"). InnerVolumeSpecName "kube-api-access-dsdnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:25:47.941209 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.938579    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsdnf\" (UniqueName: \"kubernetes.io/projected/36cee36f-9e21-4e28-9091-95f051e3a94f-kube-api-access-dsdnf\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:47.941209 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.938619    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36cee36f-9e21-4e28-9091-95f051e3a94f-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:47.949387 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.949348    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36cee36f-9e21-4e28-9091-95f051e3a94f-maas-api-tls" pod "36cee36f-9e21-4e28-9091-95f051e3a94f" (UID: "36cee36f-9e21-4e28-9091-95f051e3a94f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:25:47.964557 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:47.964519    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36cee36f-9e21-4e28-9091-95f051e3a94f-metrics-certs" pod "36cee36f-9e21-4e28-9091-95f051e3a94f" (UID: "36cee36f-9e21-4e28-9091-95f051e3a94f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:25:48.028469 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:48.028420    4524 generic.go:356] "Generic (PLEG): container finished" podID="36cee36f-9e21-4e28-9091-95f051e3a94f" containerID="1a0c7cbca8c0877881fae334b489870784e9dd5425b9ceca50be0632f71bb65f" exitCode=0
Oct 30 19:25:48.028832 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:48.028806    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4" event={"ID":"36cee36f-9e21-4e28-9091-95f051e3a94f","Type":"ContainerDied","Data":"1a0c7cbca8c0877881fae334b489870784e9dd5425b9ceca50be0632f71bb65f"}
Oct 30 19:25:48.029029 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:48.029014    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4" event={"ID":"36cee36f-9e21-4e28-9091-95f051e3a94f","Type":"ContainerDied","Data":"eee55feaebbd9e4b79831c61a756775f2293d5eeb3955afd236fc8835fbf3b30"}
Oct 30 19:25:48.029490 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:48.029471    4524 scope.go:122] "RemoveContainer" containerID="1a0c7cbca8c0877881fae334b489870784e9dd5425b9ceca50be0632f71bb65f"
Oct 30 19:25:48.029954 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:48.029934    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"
Oct 30 19:25:48.062204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:48.062141    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36cee36f-9e21-4e28-9091-95f051e3a94f-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:48.062204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:48.062194    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36cee36f-9e21-4e28-9091-95f051e3a94f-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:48.113675 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:48.113622    4524 scope.go:122] "RemoveContainer" containerID="1a0c7cbca8c0877881fae334b489870784e9dd5425b9ceca50be0632f71bb65f"
Oct 30 19:25:48.114117 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:25:48.114087    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a0c7cbca8c0877881fae334b489870784e9dd5425b9ceca50be0632f71bb65f\": container with ID starting with 1a0c7cbca8c0877881fae334b489870784e9dd5425b9ceca50be0632f71bb65f not found: ID does not exist" containerID="1a0c7cbca8c0877881fae334b489870784e9dd5425b9ceca50be0632f71bb65f"
Oct 30 19:25:48.114272 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:48.114128    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a0c7cbca8c0877881fae334b489870784e9dd5425b9ceca50be0632f71bb65f"} err="failed to get container status \"1a0c7cbca8c0877881fae334b489870784e9dd5425b9ceca50be0632f71bb65f\": rpc error: code = NotFound desc = could not find container \"1a0c7cbca8c0877881fae334b489870784e9dd5425b9ceca50be0632f71bb65f\": container with ID starting with 1a0c7cbca8c0877881fae334b489870784e9dd5425b9ceca50be0632f71bb65f not found: ID does not exist"
Oct 30 19:25:48.118655 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:48.118594    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"]
Oct 30 19:25:48.127540 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:48.127506    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4"]
Oct 30 19:25:48.561279 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:48.561249    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36cee36f-9e21-4e28-9091-95f051e3a94f" path="/var/lib/kubelet/pods/36cee36f-9e21-4e28-9091-95f051e3a94f/volumes"
Oct 30 19:25:48.594738 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:48.594709    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk"]
Oct 30 19:25:48.598294 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:25:48.597528    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod533f548c_cf0b_4bd7_9fed_2d9bde66ec95.slice/crio-7e05cefd03ac89025081b9fa745b400b8646105c0f9136d1439647a1b78a5d41 WatchSource:0}: Error finding container 7e05cefd03ac89025081b9fa745b400b8646105c0f9136d1439647a1b78a5d41: Status 404 returned error can't find the container with id 7e05cefd03ac89025081b9fa745b400b8646105c0f9136d1439647a1b78a5d41
Oct 30 19:25:48.598479 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:25:48.598438    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c50188a_2ca9_4063_9092_9fdd095582aa.slice/crio-711ce73886786c62cd4d5b87c40028689d33875d681c71800a8f19825b22d253 WatchSource:0}: Error finding container 711ce73886786c62cd4d5b87c40028689d33875d681c71800a8f19825b22d253: Status 404 returned error can't find the container with id 711ce73886786c62cd4d5b87c40028689d33875d681c71800a8f19825b22d253
Oct 30 19:25:48.599065 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:48.599043    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87"]
Oct 30 19:25:49.155534 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:49.155025    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk" event={"ID":"533f548c-cf0b-4bd7-9fed-2d9bde66ec95","Type":"ContainerStarted","Data":"82e3e1defc32af005ce07a40e50bc0289314db1694f99cc9924d6ed98b6099d0"}
Oct 30 19:25:49.155534 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:49.155091    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk" event={"ID":"533f548c-cf0b-4bd7-9fed-2d9bde66ec95","Type":"ContainerStarted","Data":"7e05cefd03ac89025081b9fa745b400b8646105c0f9136d1439647a1b78a5d41"}
Oct 30 19:25:49.155824 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:49.155801    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk"
Oct 30 19:25:49.158961 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:49.158926    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87" event={"ID":"1c50188a-2ca9-4063-9092-9fdd095582aa","Type":"ContainerStarted","Data":"5fe9601fadd8cf475b64efba769a18e4dac62beaa7e097d3cb983176d0c33000"}
Oct 30 19:25:49.159212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:49.158999    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87" event={"ID":"1c50188a-2ca9-4063-9092-9fdd095582aa","Type":"ContainerStarted","Data":"711ce73886786c62cd4d5b87c40028689d33875d681c71800a8f19825b22d253"}
Oct 30 19:25:49.159729 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:49.159676    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87"
Oct 30 19:25:49.195684 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:49.195616    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk" podStartSLOduration=2.195597858 podStartE2EDuration="2.195597858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:49.186651334 +0000 UTC m=+1180.927642416" watchObservedRunningTime="2026-10-30 19:25:49.195597858 +0000 UTC m=+1180.936588939"
Oct 30 19:25:49.211101 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:49.211042    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87" podStartSLOduration=2.211024847 podStartE2EDuration="2.211024847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:49.207125532 +0000 UTC m=+1180.948116612" watchObservedRunningTime="2026-10-30 19:25:49.211024847 +0000 UTC m=+1180.952015926"
Oct 30 19:25:50.527319 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:50.527266    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c5ed0670-84c56676f9-8mdm4" podUID="36cee36f-9e21-4e28-9091-95f051e3a94f" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.104:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:25:51.520227 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:51.520148    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5" podUID="e94b5346-79f3-494c-bfba-a72938a84194" containerName="payload-processing" containerID="cri-o://5a85b9a87b60418861d8aafbe62f422b150187f7734996562b8502cb01d3720f" gracePeriod=25
Oct 30 19:25:51.528150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:51.528117    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j" podUID="fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab" containerName="payload-pre-processing" containerID="cri-o://bbdbc8e3e6aafd6652df4428b39f53b6257596193f0a44a5c51bb4650d0c3503" gracePeriod=25
Oct 30 19:25:51.961752 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:51.961706    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5"
Oct 30 19:25:51.965153 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:51.965132    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j"
Oct 30 19:25:52.004499 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.004468    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e94b5346-79f3-494c-bfba-a72938a84194-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e94b5346-79f3-494c-bfba-a72938a84194-plugins-config-volume\") pod \"e94b5346-79f3-494c-bfba-a72938a84194\" (UID: \"e94b5346-79f3-494c-bfba-a72938a84194\") "
Oct 30 19:25:52.004686 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.004646    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e94b5346-79f3-494c-bfba-a72938a84194-kube-api-access-q589r\" (UniqueName: \"kubernetes.io/projected/e94b5346-79f3-494c-bfba-a72938a84194-kube-api-access-q589r\") pod \"e94b5346-79f3-494c-bfba-a72938a84194\" (UID: \"e94b5346-79f3-494c-bfba-a72938a84194\") "
Oct 30 19:25:52.004901 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.004696    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab-kube-api-access-gkftp\" (UniqueName: \"kubernetes.io/projected/fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab-kube-api-access-gkftp\") pod \"fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab\" (UID: \"fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab\") "
Oct 30 19:25:52.004901 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.004771    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab-plugins-config-volume\") pod \"fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab\" (UID: \"fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab\") "
Oct 30 19:25:52.005871 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.005753    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab-plugins-config-volume" pod "fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab" (UID: "fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:25:52.006482 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.006458    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e94b5346-79f3-494c-bfba-a72938a84194-plugins-config-volume" pod "e94b5346-79f3-494c-bfba-a72938a84194" (UID: "e94b5346-79f3-494c-bfba-a72938a84194"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:25:52.008062 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.008029    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e94b5346-79f3-494c-bfba-a72938a84194-kube-api-access-q589r" pod "e94b5346-79f3-494c-bfba-a72938a84194" (UID: "e94b5346-79f3-494c-bfba-a72938a84194"). InnerVolumeSpecName "kube-api-access-q589r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:25:52.009104 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.009077    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab-kube-api-access-gkftp" pod "fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab" (UID: "fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab"). InnerVolumeSpecName "kube-api-access-gkftp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:25:52.106460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.106425    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q589r\" (UniqueName: \"kubernetes.io/projected/e94b5346-79f3-494c-bfba-a72938a84194-kube-api-access-q589r\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:52.106460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.106451    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkftp\" (UniqueName: \"kubernetes.io/projected/fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab-kube-api-access-gkftp\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:52.106460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.106461    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:52.106725 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.106471    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e94b5346-79f3-494c-bfba-a72938a84194-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:52.244148 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.244115    4524 generic.go:356] "Generic (PLEG): container finished" podID="fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab" containerID="bbdbc8e3e6aafd6652df4428b39f53b6257596193f0a44a5c51bb4650d0c3503" exitCode=0
Oct 30 19:25:52.244358 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.244164    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j"
Oct 30 19:25:52.244358 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.244205    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j" event={"ID":"fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab","Type":"ContainerDied","Data":"bbdbc8e3e6aafd6652df4428b39f53b6257596193f0a44a5c51bb4650d0c3503"}
Oct 30 19:25:52.244358 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.244261    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j" event={"ID":"fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab","Type":"ContainerDied","Data":"3da6da4a9d0a31e38dba6f62ae09fb606fb0776921a87e2ddefaa73813f9c075"}
Oct 30 19:25:52.244358 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.244292    4524 scope.go:122] "RemoveContainer" containerID="bbdbc8e3e6aafd6652df4428b39f53b6257596193f0a44a5c51bb4650d0c3503"
Oct 30 19:25:52.245981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.245959    4524 generic.go:356] "Generic (PLEG): container finished" podID="e94b5346-79f3-494c-bfba-a72938a84194" containerID="5a85b9a87b60418861d8aafbe62f422b150187f7734996562b8502cb01d3720f" exitCode=0
Oct 30 19:25:52.246077 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.246014    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5"
Oct 30 19:25:52.246077 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.246034    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5" event={"ID":"e94b5346-79f3-494c-bfba-a72938a84194","Type":"ContainerDied","Data":"5a85b9a87b60418861d8aafbe62f422b150187f7734996562b8502cb01d3720f"}
Oct 30 19:25:52.246203 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.246090    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5" event={"ID":"e94b5346-79f3-494c-bfba-a72938a84194","Type":"ContainerDied","Data":"89f66d97abf5d0681cb76dd067df5867e987b5c73ef33cf85d6d8b4aee25a9df"}
Oct 30 19:25:52.256970 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.256951    4524 scope.go:122] "RemoveContainer" containerID="bbdbc8e3e6aafd6652df4428b39f53b6257596193f0a44a5c51bb4650d0c3503"
Oct 30 19:25:52.257271 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:25:52.257251    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbdbc8e3e6aafd6652df4428b39f53b6257596193f0a44a5c51bb4650d0c3503\": container with ID starting with bbdbc8e3e6aafd6652df4428b39f53b6257596193f0a44a5c51bb4650d0c3503 not found: ID does not exist" containerID="bbdbc8e3e6aafd6652df4428b39f53b6257596193f0a44a5c51bb4650d0c3503"
Oct 30 19:25:52.257360 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.257277    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbdbc8e3e6aafd6652df4428b39f53b6257596193f0a44a5c51bb4650d0c3503"} err="failed to get container status \"bbdbc8e3e6aafd6652df4428b39f53b6257596193f0a44a5c51bb4650d0c3503\": rpc error: code = NotFound desc = could not find container \"bbdbc8e3e6aafd6652df4428b39f53b6257596193f0a44a5c51bb4650d0c3503\": container with ID starting with bbdbc8e3e6aafd6652df4428b39f53b6257596193f0a44a5c51bb4650d0c3503 not found: ID does not exist"
Oct 30 19:25:52.257360 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.257293    4524 scope.go:122] "RemoveContainer" containerID="5a85b9a87b60418861d8aafbe62f422b150187f7734996562b8502cb01d3720f"
Oct 30 19:25:52.266810 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.266793    4524 scope.go:122] "RemoveContainer" containerID="5a85b9a87b60418861d8aafbe62f422b150187f7734996562b8502cb01d3720f"
Oct 30 19:25:52.267051 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:25:52.267031    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a85b9a87b60418861d8aafbe62f422b150187f7734996562b8502cb01d3720f\": container with ID starting with 5a85b9a87b60418861d8aafbe62f422b150187f7734996562b8502cb01d3720f not found: ID does not exist" containerID="5a85b9a87b60418861d8aafbe62f422b150187f7734996562b8502cb01d3720f"
Oct 30 19:25:52.267107 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.267057    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a85b9a87b60418861d8aafbe62f422b150187f7734996562b8502cb01d3720f"} err="failed to get container status \"5a85b9a87b60418861d8aafbe62f422b150187f7734996562b8502cb01d3720f\": rpc error: code = NotFound desc = could not find container \"5a85b9a87b60418861d8aafbe62f422b150187f7734996562b8502cb01d3720f\": container with ID starting with 5a85b9a87b60418861d8aafbe62f422b150187f7734996562b8502cb01d3720f not found: ID does not exist"
Oct 30 19:25:52.272548 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.272526    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5"]
Oct 30 19:25:52.279054 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.279034    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-c5ed0670-7df696cc96-8jkk5"]
Oct 30 19:25:52.287928 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.287908    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j"]
Oct 30 19:25:52.293544 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.293525    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-c5ed0670-6d99bfdc75-ftl8j"]
Oct 30 19:25:52.451414 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.451326    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e94b5346-79f3-494c-bfba-a72938a84194" path="/var/lib/kubelet/pods/e94b5346-79f3-494c-bfba-a72938a84194/volumes"
Oct 30 19:25:52.453464 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:52.452421    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab" path="/var/lib/kubelet/pods/fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab/volumes"
Oct 30 19:25:53.846546 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:53.846515    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"]
Oct 30 19:25:53.847223 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:53.847206    4524 cpu_manager.go:414] "containerMap: removing container" podUID="fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab" containerName="payload-pre-processing"
Oct 30 19:25:53.847287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:53.847224    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab" containerName="payload-pre-processing"
Oct 30 19:25:53.847287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:53.847241    4524 cpu_manager.go:414] "containerMap: removing container" podUID="e94b5346-79f3-494c-bfba-a72938a84194" containerName="payload-processing"
Oct 30 19:25:53.847287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:53.847246    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e94b5346-79f3-494c-bfba-a72938a84194" containerName="payload-processing"
Oct 30 19:25:53.847287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:53.847280    4524 cpu_manager.go:414] "containerMap: removing container" podUID="36cee36f-9e21-4e28-9091-95f051e3a94f" containerName="maas-api"
Oct 30 19:25:53.847287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:53.847287    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36cee36f-9e21-4e28-9091-95f051e3a94f" containerName="maas-api"
Oct 30 19:25:53.847460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:53.847383    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="36cee36f-9e21-4e28-9091-95f051e3a94f" containerName="maas-api"
Oct 30 19:25:53.847460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:53.847394    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa4b7e4b-b8fa-4c63-b119-4516cc9c59ab" containerName="payload-pre-processing"
Oct 30 19:25:53.847460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:53.847403    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="e94b5346-79f3-494c-bfba-a72938a84194" containerName="payload-processing"
Oct 30 19:25:53.851698 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:53.851679    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"
Oct 30 19:25:53.856267 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:53.856241    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w0-2405fb-serving-cert\""
Oct 30 19:25:53.862699 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:53.862675    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"]
Oct 30 19:25:53.943912 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:53.943874    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0183f05e-6c5a-4614-ac22-d050be53409f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b\" (UID: \"0183f05e-6c5a-4614-ac22-d050be53409f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"
Oct 30 19:25:53.943912 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:53.943930    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0183f05e-6c5a-4614-ac22-d050be53409f-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b\" (UID: \"0183f05e-6c5a-4614-ac22-d050be53409f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"
Oct 30 19:25:53.944340 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:53.944307    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmq78\" (UniqueName: \"kubernetes.io/projected/0183f05e-6c5a-4614-ac22-d050be53409f-kube-api-access-zmq78\") pod \"maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b\" (UID: \"0183f05e-6c5a-4614-ac22-d050be53409f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"
Oct 30 19:25:53.944477 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:53.944456    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0183f05e-6c5a-4614-ac22-d050be53409f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b\" (UID: \"0183f05e-6c5a-4614-ac22-d050be53409f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"
Oct 30 19:25:54.054731 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:54.049158    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmq78\" (UniqueName: \"kubernetes.io/projected/0183f05e-6c5a-4614-ac22-d050be53409f-kube-api-access-zmq78\") pod \"maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b\" (UID: \"0183f05e-6c5a-4614-ac22-d050be53409f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"
Oct 30 19:25:54.054731 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:54.049307    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0183f05e-6c5a-4614-ac22-d050be53409f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b\" (UID: \"0183f05e-6c5a-4614-ac22-d050be53409f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"
Oct 30 19:25:54.054731 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:54.049367    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0183f05e-6c5a-4614-ac22-d050be53409f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b\" (UID: \"0183f05e-6c5a-4614-ac22-d050be53409f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"
Oct 30 19:25:54.054731 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:54.049406    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0183f05e-6c5a-4614-ac22-d050be53409f-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b\" (UID: \"0183f05e-6c5a-4614-ac22-d050be53409f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"
Oct 30 19:25:54.054731 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:25:54.050199    4524 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-serving-cert: secret "maas-api-e2e-shared-a-w0-2405fb-serving-cert" not found
Oct 30 19:25:54.054731 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:25:54.050282    4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0183f05e-6c5a-4614-ac22-d050be53409f-maas-api-tls podName:0183f05e-6c5a-4614-ac22-d050be53409f nodeName:}" failed. No retries permitted until 2026-10-30 19:25:54.550259162 +0000 UTC m=+1186.291250221 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0183f05e-6c5a-4614-ac22-d050be53409f-maas-api-tls") pod "maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b" (UID: "0183f05e-6c5a-4614-ac22-d050be53409f") : secret "maas-api-e2e-shared-a-w0-2405fb-serving-cert" not found
Oct 30 19:25:54.054731 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:54.051494    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0183f05e-6c5a-4614-ac22-d050be53409f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b\" (UID: \"0183f05e-6c5a-4614-ac22-d050be53409f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"
Oct 30 19:25:54.067200 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:54.064955    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0183f05e-6c5a-4614-ac22-d050be53409f-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b\" (UID: \"0183f05e-6c5a-4614-ac22-d050be53409f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"
Oct 30 19:25:54.096228 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:54.093833    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmq78\" (UniqueName: \"kubernetes.io/projected/0183f05e-6c5a-4614-ac22-d050be53409f-kube-api-access-zmq78\") pod \"maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b\" (UID: \"0183f05e-6c5a-4614-ac22-d050be53409f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"
Oct 30 19:25:54.558041 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:54.558006    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0183f05e-6c5a-4614-ac22-d050be53409f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b\" (UID: \"0183f05e-6c5a-4614-ac22-d050be53409f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"
Oct 30 19:25:54.560480 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:54.560455    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0183f05e-6c5a-4614-ac22-d050be53409f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b\" (UID: \"0183f05e-6c5a-4614-ac22-d050be53409f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"
Oct 30 19:25:54.774201 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:54.772562    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"
Oct 30 19:25:55.157950 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.157924    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"]
Oct 30 19:25:55.161245 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:25:55.161212    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0183f05e_6c5a_4614_ac22_d050be53409f.slice/crio-26d9b7b5cb1316a2dd118bbe8adda80485854e2ba04249862a6d84aac886c499 WatchSource:0}: Error finding container 26d9b7b5cb1316a2dd118bbe8adda80485854e2ba04249862a6d84aac886c499: Status 404 returned error can't find the container with id 26d9b7b5cb1316a2dd118bbe8adda80485854e2ba04249862a6d84aac886c499
Oct 30 19:25:55.329205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.311282    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b" event={"ID":"0183f05e-6c5a-4614-ac22-d050be53409f","Type":"ContainerStarted","Data":"26d9b7b5cb1316a2dd118bbe8adda80485854e2ba04249862a6d84aac886c499"}
Oct 30 19:25:55.717997 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.717968    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d"]
Oct 30 19:25:55.724158 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.724132    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d"
Oct 30 19:25:55.729244 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.729219    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g"]
Oct 30 19:25:55.742592 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.735844    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w1-618d93-dockercfg-q2cg9\""
Oct 30 19:25:55.742592 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.735938    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-618d93\""
Oct 30 19:25:55.742592 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.736772    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g"
Oct 30 19:25:55.742592 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.740375    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d"]
Oct 30 19:25:55.751650 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.748600    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g"]
Oct 30 19:25:55.811585 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.811550    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97dda0db-86c4-4e0c-92af-55bb1e85e2f3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d\" (UID: \"97dda0db-86c4-4e0c-92af-55bb1e85e2f3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d"
Oct 30 19:25:55.811743 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.811620    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zpsd\" (UniqueName: \"kubernetes.io/projected/b7b95d1a-dbee-45f7-806f-34a1b28a7fcf-kube-api-access-9zpsd\") pod \"payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g\" (UID: \"b7b95d1a-dbee-45f7-806f-34a1b28a7fcf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g"
Oct 30 19:25:55.811822 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.811794    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfxh9\" (UniqueName: \"kubernetes.io/projected/97dda0db-86c4-4e0c-92af-55bb1e85e2f3-kube-api-access-kfxh9\") pod \"payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d\" (UID: \"97dda0db-86c4-4e0c-92af-55bb1e85e2f3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d"
Oct 30 19:25:55.811989 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.811956    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7b95d1a-dbee-45f7-806f-34a1b28a7fcf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g\" (UID: \"b7b95d1a-dbee-45f7-806f-34a1b28a7fcf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g"
Oct 30 19:25:55.950238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.943527    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7b95d1a-dbee-45f7-806f-34a1b28a7fcf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g\" (UID: \"b7b95d1a-dbee-45f7-806f-34a1b28a7fcf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g"
Oct 30 19:25:55.950238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.943642    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97dda0db-86c4-4e0c-92af-55bb1e85e2f3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d\" (UID: \"97dda0db-86c4-4e0c-92af-55bb1e85e2f3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d"
Oct 30 19:25:55.950238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.943702    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zpsd\" (UniqueName: \"kubernetes.io/projected/b7b95d1a-dbee-45f7-806f-34a1b28a7fcf-kube-api-access-9zpsd\") pod \"payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g\" (UID: \"b7b95d1a-dbee-45f7-806f-34a1b28a7fcf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g"
Oct 30 19:25:55.950238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.943852    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfxh9\" (UniqueName: \"kubernetes.io/projected/97dda0db-86c4-4e0c-92af-55bb1e85e2f3-kube-api-access-kfxh9\") pod \"payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d\" (UID: \"97dda0db-86c4-4e0c-92af-55bb1e85e2f3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d"
Oct 30 19:25:55.950238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.945191    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7b95d1a-dbee-45f7-806f-34a1b28a7fcf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g\" (UID: \"b7b95d1a-dbee-45f7-806f-34a1b28a7fcf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g"
Oct 30 19:25:55.956901 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.956867    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97dda0db-86c4-4e0c-92af-55bb1e85e2f3-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d\" (UID: \"97dda0db-86c4-4e0c-92af-55bb1e85e2f3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d"
Oct 30 19:25:55.963189 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.963123    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfxh9\" (UniqueName: \"kubernetes.io/projected/97dda0db-86c4-4e0c-92af-55bb1e85e2f3-kube-api-access-kfxh9\") pod \"payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d\" (UID: \"97dda0db-86c4-4e0c-92af-55bb1e85e2f3\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d"
Oct 30 19:25:55.999194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:55.995825    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zpsd\" (UniqueName: \"kubernetes.io/projected/b7b95d1a-dbee-45f7-806f-34a1b28a7fcf-kube-api-access-9zpsd\") pod \"payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g\" (UID: \"b7b95d1a-dbee-45f7-806f-34a1b28a7fcf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g"
Oct 30 19:25:56.080629 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:56.079429    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d"
Oct 30 19:25:56.091194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:56.082430    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g"
Oct 30 19:25:56.335944 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:56.334544    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b" event={"ID":"0183f05e-6c5a-4614-ac22-d050be53409f","Type":"ContainerStarted","Data":"43a0455957e3dae934eeb83e0a97213c64cdbcb55d11367c5bc9ff00220ad589"}
Oct 30 19:25:56.335944 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:56.335883    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"
Oct 30 19:25:56.368191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:56.364737    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b" podStartSLOduration=2.867368651 podStartE2EDuration="3.364716639s" totalImagesPullingTime="497.347988ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:56.359355107 +0000 UTC m=+1188.100346186" watchObservedRunningTime="2026-10-30 19:25:56.364716639 +0000 UTC m=+1188.105707721"
Oct 30 19:25:56.631518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:56.631447    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d"]
Oct 30 19:25:56.830366 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:25:56.830338    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7b95d1a_dbee_45f7_806f_34a1b28a7fcf.slice/crio-0fcf2a93414f1ba11fb6b081e0982f52f3be92c076abcbc87f83d6080653d312 WatchSource:0}: Error finding container 0fcf2a93414f1ba11fb6b081e0982f52f3be92c076abcbc87f83d6080653d312: Status 404 returned error can't find the container with id 0fcf2a93414f1ba11fb6b081e0982f52f3be92c076abcbc87f83d6080653d312
Oct 30 19:25:56.831101 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:56.831077    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g"]
Oct 30 19:25:57.359375 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:57.359337    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d" event={"ID":"97dda0db-86c4-4e0c-92af-55bb1e85e2f3","Type":"ContainerStarted","Data":"f76675191dc14101d26001b1232f3c4a1660d1b752ab49f1fa3d55e4072332be"}
Oct 30 19:25:57.359871 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:57.359433    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d" event={"ID":"97dda0db-86c4-4e0c-92af-55bb1e85e2f3","Type":"ContainerStarted","Data":"ea42ccd1942a73aede11d83c4d79d3412a8aa38890f1fc835ebf449cffcc439d"}
Oct 30 19:25:57.360021 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:57.359988    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d"
Oct 30 19:25:57.361360 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:57.361322    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g" event={"ID":"b7b95d1a-dbee-45f7-806f-34a1b28a7fcf","Type":"ContainerStarted","Data":"f166492b87b0df6fa0e667b36463add7d7adb6a242a321d3da61230b1720a3e8"}
Oct 30 19:25:57.361465 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:57.361390    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g"
Oct 30 19:25:57.361465 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:57.361413    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g" event={"ID":"b7b95d1a-dbee-45f7-806f-34a1b28a7fcf","Type":"ContainerStarted","Data":"0fcf2a93414f1ba11fb6b081e0982f52f3be92c076abcbc87f83d6080653d312"}
Oct 30 19:25:57.379842 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:57.379786    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d" podStartSLOduration=2.379770026 podStartE2EDuration="2.379770026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:57.375894008 +0000 UTC m=+1189.116885088" watchObservedRunningTime="2026-10-30 19:25:57.379770026 +0000 UTC m=+1189.120761104"
Oct 30 19:25:57.396833 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:25:57.396779    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g" podStartSLOduration=2.3967662770000002 podStartE2EDuration="2.396766277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:57.39274192 +0000 UTC m=+1189.133733000" watchObservedRunningTime="2026-10-30 19:25:57.396766277 +0000 UTC m=+1189.137757370"
Oct 30 19:26:00.211804 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:00.211773    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk"
Oct 30 19:26:01.235643 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:01.234972    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"]
Oct 30 19:26:01.249701 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:01.249619    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87"
Oct 30 19:26:01.251049 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:01.249934    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"
Oct 30 19:26:01.251049 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:01.250293    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"]
Oct 30 19:26:01.258125 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:01.258104    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-618d93-serving-cert\""
Oct 30 19:26:01.439540 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:01.439497    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql\" (UID: \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"
Oct 30 19:26:01.439941 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:01.439911    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql\" (UID: \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"
Oct 30 19:26:01.440124 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:01.440105    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql\" (UID: \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"
Oct 30 19:26:01.440280 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:01.440262    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwlnl\" (UniqueName: \"kubernetes.io/projected/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-kube-api-access-gwlnl\") pod \"maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql\" (UID: \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"
Oct 30 19:26:01.562249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:01.559358    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql\" (UID: \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"
Oct 30 19:26:01.562249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:01.559456    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql\" (UID: \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"
Oct 30 19:26:01.562249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:01.559501    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql\" (UID: \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"
Oct 30 19:26:01.562249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:01.559538    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwlnl\" (UniqueName: \"kubernetes.io/projected/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-kube-api-access-gwlnl\") pod \"maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql\" (UID: \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"
Oct 30 19:26:01.563813 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:01.563769    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql\" (UID: \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"
Oct 30 19:26:01.564132 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:26:01.564107    4524 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-serving-cert: secret "maas-api-e2e-shared-a-w1-618d93-serving-cert" not found
Oct 30 19:26:01.580394 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:26:01.571718    4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-maas-api-tls podName:1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd nodeName:}" failed. No retries permitted until 2026-10-30 19:26:02.071691319 +0000 UTC m=+1193.812682381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-maas-api-tls") pod "maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql" (UID: "1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd") : secret "maas-api-e2e-shared-a-w1-618d93-serving-cert" not found
Oct 30 19:26:01.580394 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:01.572217    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql\" (UID: \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"
Oct 30 19:26:01.580394 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:01.576838    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwlnl\" (UniqueName: \"kubernetes.io/projected/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-kube-api-access-gwlnl\") pod \"maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql\" (UID: \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"
Oct 30 19:26:02.076358 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:02.074708    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql\" (UID: \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"
Oct 30 19:26:02.122193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:02.113253    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql\" (UID: \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"
Oct 30 19:26:02.180084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:02.179537    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"
Oct 30 19:26:02.522426 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:02.522393    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"]
Oct 30 19:26:02.524750 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:26:02.524724    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d2a7bd3_5c2c_4e75_aac9_679a0f7c8ebd.slice/crio-c6a4ceb07164ecdd06595438b5924e29b43c2a21d09d0e4639fa7a95603c1974 WatchSource:0}: Error finding container c6a4ceb07164ecdd06595438b5924e29b43c2a21d09d0e4639fa7a95603c1974: Status 404 returned error can't find the container with id c6a4ceb07164ecdd06595438b5924e29b43c2a21d09d0e4639fa7a95603c1974
Oct 30 19:26:03.375316 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:03.375285    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"
Oct 30 19:26:03.483444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:03.469703    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql" event={"ID":"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd","Type":"ContainerStarted","Data":"7f8b518634a403388e1659c3b800eec20b4f544425c988518b164c095a851a4b"}
Oct 30 19:26:03.483746 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:03.483718    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql" event={"ID":"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd","Type":"ContainerStarted","Data":"c6a4ceb07164ecdd06595438b5924e29b43c2a21d09d0e4639fa7a95603c1974"}
Oct 30 19:26:03.483927 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:03.483912    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"
Oct 30 19:26:03.522151 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:03.522097    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql" podStartSLOduration=2.074364024 podStartE2EDuration="2.52208301s" totalImagesPullingTime="447.718986ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:03.519223444 +0000 UTC m=+1195.260214523" watchObservedRunningTime="2026-10-30 19:26:03.52208301 +0000 UTC m=+1195.263074089"
Oct 30 19:26:04.469978 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.469948    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69"]
Oct 30 19:26:04.475468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.475445    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69"
Oct 30 19:26:04.485440 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.485417    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-0d360546\""
Oct 30 19:26:04.486002 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.485952    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-0d360546-dockercfg-cdqbj\""
Oct 30 19:26:04.492544 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.491009    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69"]
Oct 30 19:26:04.501988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.500397    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl"]
Oct 30 19:26:04.520511 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.520485    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl"
Oct 30 19:26:04.532461 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.530856    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl"]
Oct 30 19:26:04.559009 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.558962    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0a36afc-349e-4a8a-8f3c-1694b6dd3510-plugins-config-volume\") pod \"payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69\" (UID: \"b0a36afc-349e-4a8a-8f3c-1694b6dd3510\") " pod="openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69"
Oct 30 19:26:04.559151 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.559111    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sv86\" (UniqueName: \"kubernetes.io/projected/b0a36afc-349e-4a8a-8f3c-1694b6dd3510-kube-api-access-7sv86\") pod \"payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69\" (UID: \"b0a36afc-349e-4a8a-8f3c-1694b6dd3510\") " pod="openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69"
Oct 30 19:26:04.662194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.661257    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwlxp\" (UniqueName: \"kubernetes.io/projected/ce4dfb91-07f4-429c-9499-153a8773e8c7-kube-api-access-bwlxp\") pod \"payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl\" (UID: \"ce4dfb91-07f4-429c-9499-153a8773e8c7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl"
Oct 30 19:26:04.662194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.661408    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0a36afc-349e-4a8a-8f3c-1694b6dd3510-plugins-config-volume\") pod \"payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69\" (UID: \"b0a36afc-349e-4a8a-8f3c-1694b6dd3510\") " pod="openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69"
Oct 30 19:26:04.662194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.661606    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sv86\" (UniqueName: \"kubernetes.io/projected/b0a36afc-349e-4a8a-8f3c-1694b6dd3510-kube-api-access-7sv86\") pod \"payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69\" (UID: \"b0a36afc-349e-4a8a-8f3c-1694b6dd3510\") " pod="openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69"
Oct 30 19:26:04.662194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.661654    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce4dfb91-07f4-429c-9499-153a8773e8c7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl\" (UID: \"ce4dfb91-07f4-429c-9499-153a8773e8c7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl"
Oct 30 19:26:04.682193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.666106    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0a36afc-349e-4a8a-8f3c-1694b6dd3510-plugins-config-volume\") pod \"payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69\" (UID: \"b0a36afc-349e-4a8a-8f3c-1694b6dd3510\") " pod="openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69"
Oct 30 19:26:04.767088 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.766961    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sv86\" (UniqueName: \"kubernetes.io/projected/b0a36afc-349e-4a8a-8f3c-1694b6dd3510-kube-api-access-7sv86\") pod \"payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69\" (UID: \"b0a36afc-349e-4a8a-8f3c-1694b6dd3510\") " pod="openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69"
Oct 30 19:26:04.768915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.768873    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwlxp\" (UniqueName: \"kubernetes.io/projected/ce4dfb91-07f4-429c-9499-153a8773e8c7-kube-api-access-bwlxp\") pod \"payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl\" (UID: \"ce4dfb91-07f4-429c-9499-153a8773e8c7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl"
Oct 30 19:26:04.770549 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.770164    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce4dfb91-07f4-429c-9499-153a8773e8c7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl\" (UID: \"ce4dfb91-07f4-429c-9499-153a8773e8c7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl"
Oct 30 19:26:04.771770 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.771724    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce4dfb91-07f4-429c-9499-153a8773e8c7-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl\" (UID: \"ce4dfb91-07f4-429c-9499-153a8773e8c7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl"
Oct 30 19:26:04.800594 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.799039    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwlxp\" (UniqueName: \"kubernetes.io/projected/ce4dfb91-07f4-429c-9499-153a8773e8c7-kube-api-access-bwlxp\") pod \"payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl\" (UID: \"ce4dfb91-07f4-429c-9499-153a8773e8c7\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl"
Oct 30 19:26:04.800594 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.799374    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69"
Oct 30 19:26:04.849906 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:04.849577    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl"
Oct 30 19:26:05.343095 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:05.342487    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl"]
Oct 30 19:26:05.350245 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:05.350145    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69"]
Oct 30 19:26:05.490196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:05.487546    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69" event={"ID":"b0a36afc-349e-4a8a-8f3c-1694b6dd3510","Type":"ContainerStarted","Data":"1b306de15aa65b9e86c97f15f37f86ca57960ba409396bf9f6007c4fa1542805"}
Oct 30 19:26:05.490196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:05.487617    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69"
Oct 30 19:26:05.490196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:05.487642    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69" event={"ID":"b0a36afc-349e-4a8a-8f3c-1694b6dd3510","Type":"ContainerStarted","Data":"5b65bd2910a63bff14c1780cb1e3e1d2f086f007aa7a3404cf00aafc14ed6744"}
Oct 30 19:26:05.490196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:05.489627    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl" event={"ID":"ce4dfb91-07f4-429c-9499-153a8773e8c7","Type":"ContainerStarted","Data":"fe768d01efc87359e78f13123dfd2e70d91d55713355ea7e01e4041ef3fa017e"}
Oct 30 19:26:05.490196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:05.489673    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl" event={"ID":"ce4dfb91-07f4-429c-9499-153a8773e8c7","Type":"ContainerStarted","Data":"0f9a8fc7dca52124506a0587e1ebcd7a7d1ea61e20a71d9feade0baccbc1dad0"}
Oct 30 19:26:05.490196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:05.489699    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl"
Oct 30 19:26:05.508234 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:05.508156    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69" podStartSLOduration=1.508143351 podStartE2EDuration="1.508143351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:05.504616428 +0000 UTC m=+1197.245607508" watchObservedRunningTime="2026-10-30 19:26:05.508143351 +0000 UTC m=+1197.249134431"
Oct 30 19:26:05.528719 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:05.528676    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl" podStartSLOduration=1.528663881 podStartE2EDuration="1.528663881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:05.524641468 +0000 UTC m=+1197.265632545" watchObservedRunningTime="2026-10-30 19:26:05.528663881 +0000 UTC m=+1197.269655005"
Oct 30 19:26:08.423590 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:08.423560    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g"
Oct 30 19:26:09.413470 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:09.413392    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d"
Oct 30 19:26:09.487004 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:09.486973    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"
Oct 30 19:26:12.138765 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:12.138733    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s"]
Oct 30 19:26:12.145485 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:12.145463    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s"
Oct 30 19:26:12.159379 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:12.159339    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w0-1e7123-dockercfg-c247v\""
Oct 30 19:26:12.185927 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:12.183814    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s"]
Oct 30 19:26:12.229734 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:12.211249    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w0-1e7123\""
Oct 30 19:26:12.252194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:12.239988    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6skj\" (UniqueName: \"kubernetes.io/projected/050d3ce5-caa8-4f50-8328-bef71892924b-kube-api-access-g6skj\") pod \"payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s\" (UID: \"050d3ce5-caa8-4f50-8328-bef71892924b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s"
Oct 30 19:26:12.252194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:12.240196    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/050d3ce5-caa8-4f50-8328-bef71892924b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s\" (UID: \"050d3ce5-caa8-4f50-8328-bef71892924b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s"
Oct 30 19:26:12.409285 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:12.402658    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6skj\" (UniqueName: \"kubernetes.io/projected/050d3ce5-caa8-4f50-8328-bef71892924b-kube-api-access-g6skj\") pod \"payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s\" (UID: \"050d3ce5-caa8-4f50-8328-bef71892924b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s"
Oct 30 19:26:12.409285 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:12.402877    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/050d3ce5-caa8-4f50-8328-bef71892924b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s\" (UID: \"050d3ce5-caa8-4f50-8328-bef71892924b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s"
Oct 30 19:26:12.409285 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:12.404024    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/050d3ce5-caa8-4f50-8328-bef71892924b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s\" (UID: \"050d3ce5-caa8-4f50-8328-bef71892924b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s"
Oct 30 19:26:12.503866 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:12.503605    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6skj\" (UniqueName: \"kubernetes.io/projected/050d3ce5-caa8-4f50-8328-bef71892924b-kube-api-access-g6skj\") pod \"payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s\" (UID: \"050d3ce5-caa8-4f50-8328-bef71892924b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s"
Oct 30 19:26:12.786426 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:12.786333    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s"
Oct 30 19:26:13.319873 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:26:13.319836    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod050d3ce5_caa8_4f50_8328_bef71892924b.slice/crio-e2b2b46cee4ce9045ca78b399b826d46d8db0e9b4e0e532d799794240387f6ac WatchSource:0}: Error finding container e2b2b46cee4ce9045ca78b399b826d46d8db0e9b4e0e532d799794240387f6ac: Status 404 returned error can't find the container with id e2b2b46cee4ce9045ca78b399b826d46d8db0e9b4e0e532d799794240387f6ac
Oct 30 19:26:13.323068 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:13.323047    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s"]
Oct 30 19:26:13.330541 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:13.330371    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"]
Oct 30 19:26:13.343058 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:13.342899    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"]
Oct 30 19:26:13.343187 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:13.343034    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"
Oct 30 19:26:13.352062 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:13.352040    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0d360546-serving-cert\""
Oct 30 19:26:13.399236 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:13.399203    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da5d19b-9f91-4d25-ab28-ffffb3475660-metrics-certs\") pod \"maas-api-e2e-mt-0d360546-847b4f977c-s4p8b\" (UID: \"9da5d19b-9f91-4d25-ab28-ffffb3475660\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"
Oct 30 19:26:13.399801 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:13.399459    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8ght\" (UniqueName: \"kubernetes.io/projected/9da5d19b-9f91-4d25-ab28-ffffb3475660-kube-api-access-f8ght\") pod \"maas-api-e2e-mt-0d360546-847b4f977c-s4p8b\" (UID: \"9da5d19b-9f91-4d25-ab28-ffffb3475660\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"
Oct 30 19:26:13.399964 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:13.399945    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9da5d19b-9f91-4d25-ab28-ffffb3475660-maas-api-tls\") pod \"maas-api-e2e-mt-0d360546-847b4f977c-s4p8b\" (UID: \"9da5d19b-9f91-4d25-ab28-ffffb3475660\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"
Oct 30 19:26:13.400037 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:13.400026    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9da5d19b-9f91-4d25-ab28-ffffb3475660-openshift-service-ca\") pod \"maas-api-e2e-mt-0d360546-847b4f977c-s4p8b\" (UID: \"9da5d19b-9f91-4d25-ab28-ffffb3475660\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"
Oct 30 19:26:13.562266 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:13.503369    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da5d19b-9f91-4d25-ab28-ffffb3475660-metrics-certs\") pod \"maas-api-e2e-mt-0d360546-847b4f977c-s4p8b\" (UID: \"9da5d19b-9f91-4d25-ab28-ffffb3475660\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"
Oct 30 19:26:13.562266 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:13.503494    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8ght\" (UniqueName: \"kubernetes.io/projected/9da5d19b-9f91-4d25-ab28-ffffb3475660-kube-api-access-f8ght\") pod \"maas-api-e2e-mt-0d360546-847b4f977c-s4p8b\" (UID: \"9da5d19b-9f91-4d25-ab28-ffffb3475660\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"
Oct 30 19:26:13.562266 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:13.503646    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9da5d19b-9f91-4d25-ab28-ffffb3475660-maas-api-tls\") pod \"maas-api-e2e-mt-0d360546-847b4f977c-s4p8b\" (UID: \"9da5d19b-9f91-4d25-ab28-ffffb3475660\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"
Oct 30 19:26:13.562266 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:13.503728    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9da5d19b-9f91-4d25-ab28-ffffb3475660-openshift-service-ca\") pod \"maas-api-e2e-mt-0d360546-847b4f977c-s4p8b\" (UID: \"9da5d19b-9f91-4d25-ab28-ffffb3475660\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"
Oct 30 19:26:13.562266 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:13.505646    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9da5d19b-9f91-4d25-ab28-ffffb3475660-openshift-service-ca\") pod \"maas-api-e2e-mt-0d360546-847b4f977c-s4p8b\" (UID: \"9da5d19b-9f91-4d25-ab28-ffffb3475660\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"
Oct 30 19:26:13.562266 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:26:13.522863    4524 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-serving-cert: secret "maas-api-e2e-mt-0d360546-serving-cert" not found
Oct 30 19:26:13.562266 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:26:13.522959    4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9da5d19b-9f91-4d25-ab28-ffffb3475660-maas-api-tls podName:9da5d19b-9f91-4d25-ab28-ffffb3475660 nodeName:}" failed. No retries permitted until 2026-10-30 19:26:14.022925628 +0000 UTC m=+1205.763916703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9da5d19b-9f91-4d25-ab28-ffffb3475660-maas-api-tls") pod "maas-api-e2e-mt-0d360546-847b4f977c-s4p8b" (UID: "9da5d19b-9f91-4d25-ab28-ffffb3475660") : secret "maas-api-e2e-mt-0d360546-serving-cert" not found
Oct 30 19:26:13.562266 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:13.532476    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da5d19b-9f91-4d25-ab28-ffffb3475660-metrics-certs\") pod \"maas-api-e2e-mt-0d360546-847b4f977c-s4p8b\" (UID: \"9da5d19b-9f91-4d25-ab28-ffffb3475660\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"
Oct 30 19:26:13.562266 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:13.546799    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8ght\" (UniqueName: \"kubernetes.io/projected/9da5d19b-9f91-4d25-ab28-ffffb3475660-kube-api-access-f8ght\") pod \"maas-api-e2e-mt-0d360546-847b4f977c-s4p8b\" (UID: \"9da5d19b-9f91-4d25-ab28-ffffb3475660\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"
Oct 30 19:26:13.671888 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:13.671850    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s" event={"ID":"050d3ce5-caa8-4f50-8328-bef71892924b","Type":"ContainerStarted","Data":"73fae5e4e2af401846687690f9896c2b2158ba01c48bf89fdc5013b73e6f017d"}
Oct 30 19:26:13.672072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:13.671909    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s" event={"ID":"050d3ce5-caa8-4f50-8328-bef71892924b","Type":"ContainerStarted","Data":"e2b2b46cee4ce9045ca78b399b826d46d8db0e9b4e0e532d799794240387f6ac"}
Oct 30 19:26:13.672821 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:13.672803    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s"
Oct 30 19:26:13.692792 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:13.692732    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s" podStartSLOduration=1.69271879 podStartE2EDuration="1.69271879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:13.690007217 +0000 UTC m=+1205.430998293" watchObservedRunningTime="2026-10-30 19:26:13.69271879 +0000 UTC m=+1205.433709869"
Oct 30 19:26:14.029303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:14.027048    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9da5d19b-9f91-4d25-ab28-ffffb3475660-maas-api-tls\") pod \"maas-api-e2e-mt-0d360546-847b4f977c-s4p8b\" (UID: \"9da5d19b-9f91-4d25-ab28-ffffb3475660\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"
Oct 30 19:26:14.050262 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:14.050135    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9da5d19b-9f91-4d25-ab28-ffffb3475660-maas-api-tls\") pod \"maas-api-e2e-mt-0d360546-847b4f977c-s4p8b\" (UID: \"9da5d19b-9f91-4d25-ab28-ffffb3475660\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"
Oct 30 19:26:14.299249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:14.294295    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"
Oct 30 19:26:14.636888 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:14.636853    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"]
Oct 30 19:26:14.637378 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:26:14.637126    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9da5d19b_9f91_4d25_ab28_ffffb3475660.slice/crio-e85ece51f9da5ce2ff41e685832f9b9b86bdfbad4ba39b65207445001834bf6b WatchSource:0}: Error finding container e85ece51f9da5ce2ff41e685832f9b9b86bdfbad4ba39b65207445001834bf6b: Status 404 returned error can't find the container with id e85ece51f9da5ce2ff41e685832f9b9b86bdfbad4ba39b65207445001834bf6b
Oct 30 19:26:14.681070 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:14.681036    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b" event={"ID":"9da5d19b-9f91-4d25-ab28-ffffb3475660","Type":"ContainerStarted","Data":"e85ece51f9da5ce2ff41e685832f9b9b86bdfbad4ba39b65207445001834bf6b"}
Oct 30 19:26:15.631416 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:26:15.631374    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:26:15.815377 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:15.815329    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b" event={"ID":"9da5d19b-9f91-4d25-ab28-ffffb3475660","Type":"ContainerStarted","Data":"6ee07715aea6c1ca5129bc54662f23542355b9ff010bc985be639c7a60a8ddf0"}
Oct 30 19:26:15.822356 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:15.822316    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"
Oct 30 19:26:15.860264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:15.858980    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b" podStartSLOduration=2.416197145 podStartE2EDuration="2.858960793s" totalImagesPullingTime="442.763648ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:15.84897907 +0000 UTC m=+1207.589970151" watchObservedRunningTime="2026-10-30 19:26:15.858960793 +0000 UTC m=+1207.599951872"
Oct 30 19:26:16.579826 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:16.579799    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69"
Oct 30 19:26:16.582408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:16.582385    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl"
Oct 30 19:26:20.756250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:20.749218    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"]
Oct 30 19:26:20.892581 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:20.892545    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"
Oct 30 19:26:20.922811 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:20.922774    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w0-1e7123-serving-cert\""
Oct 30 19:26:21.026876 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:21.026848    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"]
Oct 30 19:26:21.032445 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:21.032148    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v\" (UID: \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"
Oct 30 19:26:21.032703 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:21.032678    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v\" (UID: \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"
Oct 30 19:26:21.032933 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:21.032895    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbxhm\" (UniqueName: \"kubernetes.io/projected/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-kube-api-access-qbxhm\") pod \"maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v\" (UID: \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"
Oct 30 19:26:21.033232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:21.033197    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v\" (UID: \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"
Oct 30 19:26:21.141659 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:21.141604    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v\" (UID: \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"
Oct 30 19:26:21.155189 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:21.151225    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v\" (UID: \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"
Oct 30 19:26:21.155189 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:21.150330    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v\" (UID: \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"
Oct 30 19:26:21.155189 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:21.151402    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbxhm\" (UniqueName: \"kubernetes.io/projected/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-kube-api-access-qbxhm\") pod \"maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v\" (UID: \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"
Oct 30 19:26:21.155189 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:21.151683    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v\" (UID: \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"
Oct 30 19:26:21.155189 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:26:21.152086    4524 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-serving-cert: secret "maas-api-e2e-shared-b-w0-1e7123-serving-cert" not found
Oct 30 19:26:21.155189 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:26:21.152151    4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-maas-api-tls podName:c23cec30-c930-46cc-a1c0-2c4f347dfcd7 nodeName:}" failed. No retries permitted until 2026-10-30 19:26:21.652135559 +0000 UTC m=+1213.393126616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-maas-api-tls") pod "maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v" (UID: "c23cec30-c930-46cc-a1c0-2c4f347dfcd7") : secret "maas-api-e2e-shared-b-w0-1e7123-serving-cert" not found
Oct 30 19:26:21.223450 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:21.223389    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbxhm\" (UniqueName: \"kubernetes.io/projected/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-kube-api-access-qbxhm\") pod \"maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v\" (UID: \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"
Oct 30 19:26:21.258148 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:21.257905    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v\" (UID: \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"
Oct 30 19:26:21.734719 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:21.734670    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v\" (UID: \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"
Oct 30 19:26:21.763954 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:21.763910    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v\" (UID: \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"
Oct 30 19:26:21.827126 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:21.827095    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"
Oct 30 19:26:21.944044 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:21.944009    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"
Oct 30 19:26:22.337310 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:22.335597    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"]
Oct 30 19:26:23.130828 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:23.130788    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v" event={"ID":"c23cec30-c930-46cc-a1c0-2c4f347dfcd7","Type":"ContainerStarted","Data":"652679f7f0f151f7210f9842af5908b7a81204141bf0d136dbc66975c7935f8d"}
Oct 30 19:26:23.131519 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:23.131490    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v" event={"ID":"c23cec30-c930-46cc-a1c0-2c4f347dfcd7","Type":"ContainerStarted","Data":"1889a1e5e35aedd4518e2e3d3e689a6e1200641abe03a2c1ccf17b4dbf753e4e"}
Oct 30 19:26:23.132955 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:23.132934    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"
Oct 30 19:26:25.817882 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:25.817852    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s"
Oct 30 19:26:25.842389 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:25.842336    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v" podStartSLOduration=5.423124076 podStartE2EDuration="5.842321265s" totalImagesPullingTime="419.197189ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:23.162857747 +0000 UTC m=+1214.903848841" watchObservedRunningTime="2026-10-30 19:26:25.842321265 +0000 UTC m=+1217.583312344"
Oct 30 19:26:29.138303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.138262    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"]
Oct 30 19:26:29.148535 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.148508    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"
Oct 30 19:26:29.173331 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.173295    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"]
Oct 30 19:26:29.219189 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.218265    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"]
Oct 30 19:26:29.227038 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.227008    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-e3586f-serving-cert\""
Oct 30 19:26:29.232502 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.232478    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"]
Oct 30 19:26:29.232628 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.232571    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.258643 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.258613    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/657a30cd-92e3-449c-971b-b3e0ffe929cc-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47\" (UID: \"657a30cd-92e3-449c-971b-b3e0ffe929cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"
Oct 30 19:26:29.258799 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.258663    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/657a30cd-92e3-449c-971b-b3e0ffe929cc-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47\" (UID: \"657a30cd-92e3-449c-971b-b3e0ffe929cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"
Oct 30 19:26:29.258799 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.258727    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/657a30cd-92e3-449c-971b-b3e0ffe929cc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47\" (UID: \"657a30cd-92e3-449c-971b-b3e0ffe929cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"
Oct 30 19:26:29.258925 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.258796    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f7zm\" (UniqueName: \"kubernetes.io/projected/657a30cd-92e3-449c-971b-b3e0ffe929cc-kube-api-access-5f7zm\") pod \"maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47\" (UID: \"657a30cd-92e3-449c-971b-b3e0ffe929cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"
Oct 30 19:26:29.360673 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.360637    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/657a30cd-92e3-449c-971b-b3e0ffe929cc-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47\" (UID: \"657a30cd-92e3-449c-971b-b3e0ffe929cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"
Oct 30 19:26:29.360839 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.360688    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/657a30cd-92e3-449c-971b-b3e0ffe929cc-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47\" (UID: \"657a30cd-92e3-449c-971b-b3e0ffe929cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"
Oct 30 19:26:29.360839 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.360732    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.360839 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.360799    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.360974 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.360857    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/657a30cd-92e3-449c-971b-b3e0ffe929cc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47\" (UID: \"657a30cd-92e3-449c-971b-b3e0ffe929cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"
Oct 30 19:26:29.360974 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.360913    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.361120 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.361095    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f7zm\" (UniqueName: \"kubernetes.io/projected/657a30cd-92e3-449c-971b-b3e0ffe929cc-kube-api-access-5f7zm\") pod \"maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47\" (UID: \"657a30cd-92e3-449c-971b-b3e0ffe929cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"
Oct 30 19:26:29.361191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.361148    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glq5g\" (UniqueName: \"kubernetes.io/projected/80627fa4-301a-4dba-bc98-6fab70d9707c-kube-api-access-glq5g\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.361300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.361256    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.361454 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.361433    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.361531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.361483    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/80627fa4-301a-4dba-bc98-6fab70d9707c-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.361648 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.361630    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/657a30cd-92e3-449c-971b-b3e0ffe929cc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47\" (UID: \"657a30cd-92e3-449c-971b-b3e0ffe929cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"
Oct 30 19:26:29.363220 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.363203    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/657a30cd-92e3-449c-971b-b3e0ffe929cc-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47\" (UID: \"657a30cd-92e3-449c-971b-b3e0ffe929cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"
Oct 30 19:26:29.363288 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.363266    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/657a30cd-92e3-449c-971b-b3e0ffe929cc-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47\" (UID: \"657a30cd-92e3-449c-971b-b3e0ffe929cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"
Oct 30 19:26:29.403221 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.403124    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f7zm\" (UniqueName: \"kubernetes.io/projected/657a30cd-92e3-449c-971b-b3e0ffe929cc-kube-api-access-5f7zm\") pod \"maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47\" (UID: \"657a30cd-92e3-449c-971b-b3e0ffe929cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"
Oct 30 19:26:29.474197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.471284    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.474197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.471432    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.474197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.471468    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/80627fa4-301a-4dba-bc98-6fab70d9707c-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.474197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.471596    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.474197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.471630    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.474197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.471685    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.474197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.471960    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glq5g\" (UniqueName: \"kubernetes.io/projected/80627fa4-301a-4dba-bc98-6fab70d9707c-kube-api-access-glq5g\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.486196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.481414    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"
Oct 30 19:26:29.486196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.481442    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/80627fa4-301a-4dba-bc98-6fab70d9707c-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.486196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.482945    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.563437 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.531581    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.563437 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.543602    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.563437 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.559873    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glq5g\" (UniqueName: \"kubernetes.io/projected/80627fa4-301a-4dba-bc98-6fab70d9707c-kube-api-access-glq5g\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.563437 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.563355    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.606234 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.596710    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-b7qjj\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:29.852205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:29.852154    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:30.104267 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:30.104236    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"]
Oct 30 19:26:30.186072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:30.186045    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"
Oct 30 19:26:30.359276 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:30.359193    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47" event={"ID":"657a30cd-92e3-449c-971b-b3e0ffe929cc","Type":"ContainerStarted","Data":"68fb54edf42e2f6b0356decbe6f22755506032c01ffa2b91ce1088f86f241ec6"}
Oct 30 19:26:30.362318 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:26:30.362282    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80627fa4_301a_4dba_bc98_6fab70d9707c.slice/crio-05a741c609695059451ea83e346a11d3eed508b55cfc2bc653df03afc8724146 WatchSource:0}: Error finding container 05a741c609695059451ea83e346a11d3eed508b55cfc2bc653df03afc8724146: Status 404 returned error can't find the container with id 05a741c609695059451ea83e346a11d3eed508b55cfc2bc653df03afc8724146
Oct 30 19:26:30.387127 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:30.368088    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:26:30.387127 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:30.368188    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:26:30.387127 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:30.370034    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"]
Oct 30 19:26:31.372073 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:31.372029    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47" event={"ID":"657a30cd-92e3-449c-971b-b3e0ffe929cc","Type":"ContainerStarted","Data":"42b2df0a5bc427e6877b5d6175ff419515a9cb1d18b46b59f873154366555914"}
Oct 30 19:26:31.372719 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:31.372657    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"
Oct 30 19:26:31.375807 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:31.375723    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj" event={"ID":"80627fa4-301a-4dba-bc98-6fab70d9707c","Type":"ContainerStarted","Data":"064e8faa05be45264206fbe7235b17b811b4fa586d058196e9fc22e8867e2976"}
Oct 30 19:26:31.375807 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:31.375775    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj" event={"ID":"80627fa4-301a-4dba-bc98-6fab70d9707c","Type":"ContainerStarted","Data":"05a741c609695059451ea83e346a11d3eed508b55cfc2bc653df03afc8724146"}
Oct 30 19:26:31.375979 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:31.375874    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:31.378217 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:31.378192    4524 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-b7qjj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:26:31.378318 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:31.378243    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj" podUID="80627fa4-301a-4dba-bc98-6fab70d9707c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:26:31.411862 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:31.411796    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47" podStartSLOduration=1.93069534 podStartE2EDuration="2.411778873s" totalImagesPullingTime="481.083533ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:31.397602267 +0000 UTC m=+1223.138593350" watchObservedRunningTime="2026-10-30 19:26:31.411778873 +0000 UTC m=+1223.152769953"
Oct 30 19:26:31.446327 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:31.446278    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj" podStartSLOduration=2.446265527 podStartE2EDuration="2.446265527s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:31.441018309 +0000 UTC m=+1223.182009382" watchObservedRunningTime="2026-10-30 19:26:31.446265527 +0000 UTC m=+1223.187256606"
Oct 30 19:26:32.398799 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:32.398770    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:26:37.390919 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:26:37.390885    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"
Oct 30 19:26:45.627503 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:26:45.627460    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:27:15.714227 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:27:15.711407    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:27:41.698981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.698950    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"]
Oct 30 19:27:41.703904 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.703881    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.722742 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.722719    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"]
Oct 30 19:27:41.785552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.785516    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-token\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.785790 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.785570    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-credential-socket\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.785790 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.785697    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-podinfo\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.785790 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.785777    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-workload-certs\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.786013 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.785847    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4m2m\" (UniqueName: \"kubernetes.io/projected/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-kube-api-access-g4m2m\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.786013 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.785949    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-workload-socket\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.786013 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.785987    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-envoy\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.786234 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.786082    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-data\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.786234 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.786127    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istiod-ca-cert\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.791775 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.791744    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-b281d5-openshift-default-dockercfg-tjwsd\""
Oct 30 19:27:41.890428 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.890381    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-workload-socket\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.898104 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.891032    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-workload-socket\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.898104 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.891186    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-envoy\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.899333 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.899308    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-data\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.899532 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.899516    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istiod-ca-cert\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.899665 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.899650    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-token\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.899791 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.899774    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-credential-socket\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.900035 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.900016    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-podinfo\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.900333 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.900314    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-workload-certs\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.900537 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.900497    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4m2m\" (UniqueName: \"kubernetes.io/projected/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-kube-api-access-g4m2m\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.901715 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.901665    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-data\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.902794 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.902437    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-credential-socket\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.903242 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.903222    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-workload-certs\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.907529 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.907505    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istiod-ca-cert\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.934440 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.934298    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-envoy\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.948037 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.947967    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4m2m\" (UniqueName: \"kubernetes.io/projected/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-kube-api-access-g4m2m\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.953563 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.953483    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-podinfo\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:41.962852 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:41.962816    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-token\") pod \"e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") " pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:42.034684 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:42.034646    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:42.328238 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:27:42.328206    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedbb4bd8_4e56_4119_9170_f2e76cdb1eb2.slice/crio-104c4322c0370f4cf3da116945ecb5a759889e891bb893f07f736252e0710663 WatchSource:0}: Error finding container 104c4322c0370f4cf3da116945ecb5a759889e891bb893f07f736252e0710663: Status 404 returned error can't find the container with id 104c4322c0370f4cf3da116945ecb5a759889e891bb893f07f736252e0710663
Oct 30 19:27:42.328797 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:42.328775    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"]
Oct 30 19:27:42.331339 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:42.331296    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:27:42.331339 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:42.331383    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:27:42.331339 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:42.331426    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:27:42.829887 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:42.829846    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h" event={"ID":"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2","Type":"ContainerStarted","Data":"eb0dcad21c73806d6e9f49c305fc9b367e3d2d4ef2afea5e3f908e24c57d0fd2"}
Oct 30 19:27:42.830398 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:42.829912    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h" event={"ID":"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2","Type":"ContainerStarted","Data":"104c4322c0370f4cf3da116945ecb5a759889e891bb893f07f736252e0710663"}
Oct 30 19:27:42.857096 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:42.857046    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h" podStartSLOduration=1.857016191 podStartE2EDuration="1.857016191s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:42.853881079 +0000 UTC m=+1294.594872168" watchObservedRunningTime="2026-10-30 19:27:42.857016191 +0000 UTC m=+1294.598007271"
Oct 30 19:27:43.034963 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:43.034930    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:43.036397 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:43.036372    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:27:43.036516 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:43.036419    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h" podUID="edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 19:27:44.035482 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:44.035446    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:27:44.035906 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:44.035509    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h" podUID="edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 19:27:45.034973 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:45.034941    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:27:45.035154 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:45.034995    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h" podUID="edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 19:27:45.634851 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:27:45.634817    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:27:46.039413 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:46.039318    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:27:46.039711 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:46.039678    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h" podUID="edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 19:27:47.038417 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:47.037702    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:27:47.038417 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:47.037768    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h" podUID="edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 19:27:48.036095 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:48.036056    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused" start-of-body=
Oct 30 19:27:48.036340 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:48.036125    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h" podUID="edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.119:15021/healthz/ready\": dial tcp 10.133.0.119:15021: connect: connection refused"
Oct 30 19:27:49.040681 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:49.040644    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:49.041124 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:49.041105    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:49.041777 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:49.041758    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:27:50.114434 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:50.114398    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"]
Oct 30 19:27:50.121578 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:50.121510    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"
Oct 30 19:27:50.127449 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:50.127373    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-9b2e27-serving-cert\""
Oct 30 19:27:50.135245 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:50.135215    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"]
Oct 30 19:27:50.164743 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:50.163068    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/299c6068-7c79-4610-bfe2-894dd531c068-maas-api-tls\") pod \"maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p\" (UID: \"299c6068-7c79-4610-bfe2-894dd531c068\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"
Oct 30 19:27:50.164743 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:50.163236    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thtpw\" (UniqueName: \"kubernetes.io/projected/299c6068-7c79-4610-bfe2-894dd531c068-kube-api-access-thtpw\") pod \"maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p\" (UID: \"299c6068-7c79-4610-bfe2-894dd531c068\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"
Oct 30 19:27:50.164743 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:50.163348    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/299c6068-7c79-4610-bfe2-894dd531c068-openshift-service-ca\") pod \"maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p\" (UID: \"299c6068-7c79-4610-bfe2-894dd531c068\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"
Oct 30 19:27:50.164743 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:50.163619    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/299c6068-7c79-4610-bfe2-894dd531c068-metrics-certs\") pod \"maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p\" (UID: \"299c6068-7c79-4610-bfe2-894dd531c068\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"
Oct 30 19:27:50.293371 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:50.280603    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/299c6068-7c79-4610-bfe2-894dd531c068-metrics-certs\") pod \"maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p\" (UID: \"299c6068-7c79-4610-bfe2-894dd531c068\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"
Oct 30 19:27:50.293371 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:50.280741    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/299c6068-7c79-4610-bfe2-894dd531c068-maas-api-tls\") pod \"maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p\" (UID: \"299c6068-7c79-4610-bfe2-894dd531c068\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"
Oct 30 19:27:50.293371 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:50.280807    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thtpw\" (UniqueName: \"kubernetes.io/projected/299c6068-7c79-4610-bfe2-894dd531c068-kube-api-access-thtpw\") pod \"maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p\" (UID: \"299c6068-7c79-4610-bfe2-894dd531c068\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"
Oct 30 19:27:50.293371 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:50.280867    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/299c6068-7c79-4610-bfe2-894dd531c068-openshift-service-ca\") pod \"maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p\" (UID: \"299c6068-7c79-4610-bfe2-894dd531c068\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"
Oct 30 19:27:50.293371 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:50.281968    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/299c6068-7c79-4610-bfe2-894dd531c068-openshift-service-ca\") pod \"maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p\" (UID: \"299c6068-7c79-4610-bfe2-894dd531c068\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"
Oct 30 19:27:50.293371 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:27:50.284984    4524 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-serving-cert: secret "maas-api-e2e-api-b-9b2e27-serving-cert" not found
Oct 30 19:27:50.293371 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:27:50.285075    4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/299c6068-7c79-4610-bfe2-894dd531c068-maas-api-tls podName:299c6068-7c79-4610-bfe2-894dd531c068 nodeName:}" failed. No retries permitted until 2026-10-30 19:27:50.785049026 +0000 UTC m=+1302.526040103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/299c6068-7c79-4610-bfe2-894dd531c068-maas-api-tls") pod "maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p" (UID: "299c6068-7c79-4610-bfe2-894dd531c068") : secret "maas-api-e2e-api-b-9b2e27-serving-cert" not found
Oct 30 19:27:50.347533 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:50.347437    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thtpw\" (UniqueName: \"kubernetes.io/projected/299c6068-7c79-4610-bfe2-894dd531c068-kube-api-access-thtpw\") pod \"maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p\" (UID: \"299c6068-7c79-4610-bfe2-894dd531c068\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"
Oct 30 19:27:50.351802 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:50.351741    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/299c6068-7c79-4610-bfe2-894dd531c068-metrics-certs\") pod \"maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p\" (UID: \"299c6068-7c79-4610-bfe2-894dd531c068\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"
Oct 30 19:27:50.801360 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:50.801324    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/299c6068-7c79-4610-bfe2-894dd531c068-maas-api-tls\") pod \"maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p\" (UID: \"299c6068-7c79-4610-bfe2-894dd531c068\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"
Oct 30 19:27:50.803739 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:50.803713    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/299c6068-7c79-4610-bfe2-894dd531c068-maas-api-tls\") pod \"maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p\" (UID: \"299c6068-7c79-4610-bfe2-894dd531c068\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"
Oct 30 19:27:51.054250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:51.054133    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"
Oct 30 19:27:51.458952 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:51.458921    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"]
Oct 30 19:27:51.972292 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:51.972256    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p" event={"ID":"299c6068-7c79-4610-bfe2-894dd531c068","Type":"ContainerStarted","Data":"d960fd013cc3b29f47f06d969ce0bee60c6ca2d36cd497041414b0b3352243c1"}
Oct 30 19:27:51.972466 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:51.972316    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p" event={"ID":"299c6068-7c79-4610-bfe2-894dd531c068","Type":"ContainerStarted","Data":"3e9c37122f0a0a7f79806ce6702237d6590876f3979d30324bf146067db801c0"}
Oct 30 19:27:51.972466 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:51.972378    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"
Oct 30 19:27:52.004758 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:52.004706    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p" podStartSLOduration=1.605580957 podStartE2EDuration="2.004691527s" totalImagesPullingTime="399.11057ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:52.00148133 +0000 UTC m=+1303.742472413" watchObservedRunningTime="2026-10-30 19:27:52.004691527 +0000 UTC m=+1303.745682605"
Oct 30 19:27:52.745221 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:52.745130    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"]
Oct 30 19:27:52.753450 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:52.753425    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"
Oct 30 19:27:52.771771 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:52.771745    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-8f3ff4-serving-cert\""
Oct 30 19:27:52.783895 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:52.783868    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"]
Oct 30 19:27:52.876258 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:52.876228    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv\" (UID: \"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"
Oct 30 19:27:52.876465 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:52.876271    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv\" (UID: \"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"
Oct 30 19:27:52.876521 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:52.876480    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv\" (UID: \"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"
Oct 30 19:27:52.876521 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:52.876511    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jf97\" (UniqueName: \"kubernetes.io/projected/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-kube-api-access-2jf97\") pod \"maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv\" (UID: \"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"
Oct 30 19:27:53.000241 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:52.996361    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv\" (UID: \"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"
Oct 30 19:27:53.000241 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:52.996425    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jf97\" (UniqueName: \"kubernetes.io/projected/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-kube-api-access-2jf97\") pod \"maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv\" (UID: \"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"
Oct 30 19:27:53.000241 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:52.996690    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv\" (UID: \"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"
Oct 30 19:27:53.000241 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:52.996727    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv\" (UID: \"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"
Oct 30 19:27:53.024197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:53.010856    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv\" (UID: \"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"
Oct 30 19:27:53.049194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:53.038304    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv\" (UID: \"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"
Oct 30 19:27:53.049194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:53.041041    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv\" (UID: \"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"
Oct 30 19:27:53.049194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:53.041214    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jf97\" (UniqueName: \"kubernetes.io/projected/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-kube-api-access-2jf97\") pod \"maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv\" (UID: \"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"
Oct 30 19:27:53.096495 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:53.096451    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"
Oct 30 19:27:53.629791 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:53.627705    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"]
Oct 30 19:27:54.052257 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:54.052154    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv" event={"ID":"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e","Type":"ContainerStarted","Data":"bc3eebe90a198f0302c738a880fd5f134b4c91aef2eedeba8e8aa1ee35733b51"}
Oct 30 19:27:55.063572 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:55.063533    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv" event={"ID":"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e","Type":"ContainerStarted","Data":"f55435c4db2701d21a2cc382511a52433e80a0e5e5f5feccc999babd1938cf21"}
Oct 30 19:27:55.064017 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:55.063749    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"
Oct 30 19:27:55.088032 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:55.087976    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv" podStartSLOduration=2.642718355 podStartE2EDuration="3.087962559s" totalImagesPullingTime="445.244204ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:27:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:27:55.083075821 +0000 UTC m=+1306.824066903" watchObservedRunningTime="2026-10-30 19:27:55.087962559 +0000 UTC m=+1306.828953638"
Oct 30 19:27:58.049482 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:27:58.049454    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"
Oct 30 19:28:01.102501 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:01.102473    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"
Oct 30 19:28:10.783986 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:10.783950    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69"]
Oct 30 19:28:10.799353 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:10.799312    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl"]
Oct 30 19:28:15.655197 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:28:15.647390    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:28:15.791190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:15.787472    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69" podUID="b0a36afc-349e-4a8a-8f3c-1694b6dd3510" containerName="payload-processing" containerID="cri-o://1b306de15aa65b9e86c97f15f37f86ca57960ba409396bf9f6007c4fa1542805" gracePeriod=25
Oct 30 19:28:15.804752 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:15.802249    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl" podUID="ce4dfb91-07f4-429c-9499-153a8773e8c7" containerName="payload-pre-processing" containerID="cri-o://fe768d01efc87359e78f13123dfd2e70d91d55713355ea7e01e4041ef3fa017e" gracePeriod=25
Oct 30 19:28:16.406936 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.406905    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69"
Oct 30 19:28:16.453446 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.453420    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl"
Oct 30 19:28:16.501150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.501071    4524 generic.go:356] "Generic (PLEG): container finished" podID="b0a36afc-349e-4a8a-8f3c-1694b6dd3510" containerID="1b306de15aa65b9e86c97f15f37f86ca57960ba409396bf9f6007c4fa1542805" exitCode=0
Oct 30 19:28:16.501342 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.501162    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69"
Oct 30 19:28:16.501342 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.501199    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69" event={"ID":"b0a36afc-349e-4a8a-8f3c-1694b6dd3510","Type":"ContainerDied","Data":"1b306de15aa65b9e86c97f15f37f86ca57960ba409396bf9f6007c4fa1542805"}
Oct 30 19:28:16.501342 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.501269    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69" event={"ID":"b0a36afc-349e-4a8a-8f3c-1694b6dd3510","Type":"ContainerDied","Data":"5b65bd2910a63bff14c1780cb1e3e1d2f086f007aa7a3404cf00aafc14ed6744"}
Oct 30 19:28:16.501342 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.501300    4524 scope.go:122] "RemoveContainer" containerID="1b306de15aa65b9e86c97f15f37f86ca57960ba409396bf9f6007c4fa1542805"
Oct 30 19:28:16.503571 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.503539    4524 generic.go:356] "Generic (PLEG): container finished" podID="ce4dfb91-07f4-429c-9499-153a8773e8c7" containerID="fe768d01efc87359e78f13123dfd2e70d91d55713355ea7e01e4041ef3fa017e" exitCode=0
Oct 30 19:28:16.503703 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.503685    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl" event={"ID":"ce4dfb91-07f4-429c-9499-153a8773e8c7","Type":"ContainerDied","Data":"fe768d01efc87359e78f13123dfd2e70d91d55713355ea7e01e4041ef3fa017e"}
Oct 30 19:28:16.503753 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.503735    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl" event={"ID":"ce4dfb91-07f4-429c-9499-153a8773e8c7","Type":"ContainerDied","Data":"0f9a8fc7dca52124506a0587e1ebcd7a7d1ea61e20a71d9feade0baccbc1dad0"}
Oct 30 19:28:16.503823 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.503809    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl"
Oct 30 19:28:16.516494 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.516417    4524 scope.go:122] "RemoveContainer" containerID="1b306de15aa65b9e86c97f15f37f86ca57960ba409396bf9f6007c4fa1542805"
Oct 30 19:28:16.516759 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:28:16.516730    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b306de15aa65b9e86c97f15f37f86ca57960ba409396bf9f6007c4fa1542805\": container with ID starting with 1b306de15aa65b9e86c97f15f37f86ca57960ba409396bf9f6007c4fa1542805 not found: ID does not exist" containerID="1b306de15aa65b9e86c97f15f37f86ca57960ba409396bf9f6007c4fa1542805"
Oct 30 19:28:16.516866 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.516768    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b306de15aa65b9e86c97f15f37f86ca57960ba409396bf9f6007c4fa1542805"} err="failed to get container status \"1b306de15aa65b9e86c97f15f37f86ca57960ba409396bf9f6007c4fa1542805\": rpc error: code = NotFound desc = could not find container \"1b306de15aa65b9e86c97f15f37f86ca57960ba409396bf9f6007c4fa1542805\": container with ID starting with 1b306de15aa65b9e86c97f15f37f86ca57960ba409396bf9f6007c4fa1542805 not found: ID does not exist"
Oct 30 19:28:16.516866 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.516791    4524 scope.go:122] "RemoveContainer" containerID="fe768d01efc87359e78f13123dfd2e70d91d55713355ea7e01e4041ef3fa017e"
Oct 30 19:28:16.521107 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.521083    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce4dfb91-07f4-429c-9499-153a8773e8c7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce4dfb91-07f4-429c-9499-153a8773e8c7-plugins-config-volume\") pod \"ce4dfb91-07f4-429c-9499-153a8773e8c7\" (UID: \"ce4dfb91-07f4-429c-9499-153a8773e8c7\") "
Oct 30 19:28:16.521303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.521279    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0a36afc-349e-4a8a-8f3c-1694b6dd3510-kube-api-access-7sv86\" (UniqueName: \"kubernetes.io/projected/b0a36afc-349e-4a8a-8f3c-1694b6dd3510-kube-api-access-7sv86\") pod \"b0a36afc-349e-4a8a-8f3c-1694b6dd3510\" (UID: \"b0a36afc-349e-4a8a-8f3c-1694b6dd3510\") "
Oct 30 19:28:16.521421 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.521328    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce4dfb91-07f4-429c-9499-153a8773e8c7-kube-api-access-bwlxp\" (UniqueName: \"kubernetes.io/projected/ce4dfb91-07f4-429c-9499-153a8773e8c7-kube-api-access-bwlxp\") pod \"ce4dfb91-07f4-429c-9499-153a8773e8c7\" (UID: \"ce4dfb91-07f4-429c-9499-153a8773e8c7\") "
Oct 30 19:28:16.521482 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.521459    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b0a36afc-349e-4a8a-8f3c-1694b6dd3510-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0a36afc-349e-4a8a-8f3c-1694b6dd3510-plugins-config-volume\") pod \"b0a36afc-349e-4a8a-8f3c-1694b6dd3510\" (UID: \"b0a36afc-349e-4a8a-8f3c-1694b6dd3510\") "
Oct 30 19:28:16.522947 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.522923    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0a36afc-349e-4a8a-8f3c-1694b6dd3510-plugins-config-volume" pod "b0a36afc-349e-4a8a-8f3c-1694b6dd3510" (UID: "b0a36afc-349e-4a8a-8f3c-1694b6dd3510"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:28:16.523410 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.523388    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce4dfb91-07f4-429c-9499-153a8773e8c7-plugins-config-volume" pod "ce4dfb91-07f4-429c-9499-153a8773e8c7" (UID: "ce4dfb91-07f4-429c-9499-153a8773e8c7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:28:16.525515 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.525484    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0a36afc-349e-4a8a-8f3c-1694b6dd3510-kube-api-access-7sv86" pod "b0a36afc-349e-4a8a-8f3c-1694b6dd3510" (UID: "b0a36afc-349e-4a8a-8f3c-1694b6dd3510"). InnerVolumeSpecName "kube-api-access-7sv86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:28:16.526217 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.526160    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce4dfb91-07f4-429c-9499-153a8773e8c7-kube-api-access-bwlxp" pod "ce4dfb91-07f4-429c-9499-153a8773e8c7" (UID: "ce4dfb91-07f4-429c-9499-153a8773e8c7"). InnerVolumeSpecName "kube-api-access-bwlxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:28:16.530794 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.530776    4524 scope.go:122] "RemoveContainer" containerID="fe768d01efc87359e78f13123dfd2e70d91d55713355ea7e01e4041ef3fa017e"
Oct 30 19:28:16.531052 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:28:16.531030    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe768d01efc87359e78f13123dfd2e70d91d55713355ea7e01e4041ef3fa017e\": container with ID starting with fe768d01efc87359e78f13123dfd2e70d91d55713355ea7e01e4041ef3fa017e not found: ID does not exist" containerID="fe768d01efc87359e78f13123dfd2e70d91d55713355ea7e01e4041ef3fa017e"
Oct 30 19:28:16.531145 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.531057    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe768d01efc87359e78f13123dfd2e70d91d55713355ea7e01e4041ef3fa017e"} err="failed to get container status \"fe768d01efc87359e78f13123dfd2e70d91d55713355ea7e01e4041ef3fa017e\": rpc error: code = NotFound desc = could not find container \"fe768d01efc87359e78f13123dfd2e70d91d55713355ea7e01e4041ef3fa017e\": container with ID starting with fe768d01efc87359e78f13123dfd2e70d91d55713355ea7e01e4041ef3fa017e not found: ID does not exist"
Oct 30 19:28:16.624040 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.624005    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce4dfb91-07f4-429c-9499-153a8773e8c7-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:16.624040 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.624035    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sv86\" (UniqueName: \"kubernetes.io/projected/b0a36afc-349e-4a8a-8f3c-1694b6dd3510-kube-api-access-7sv86\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:16.624258 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.624050    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwlxp\" (UniqueName: \"kubernetes.io/projected/ce4dfb91-07f4-429c-9499-153a8773e8c7-kube-api-access-bwlxp\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:16.624258 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.624063    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0a36afc-349e-4a8a-8f3c-1694b6dd3510-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:16.844585 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.844557    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69"]
Oct 30 19:28:16.849056 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.849033    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-0d360546-75f95c9c6-9fh69"]
Oct 30 19:28:16.868899 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.868876    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl"]
Oct 30 19:28:16.880208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:16.880182    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-0d360546-569bbdf84c-7dkbl"]
Oct 30 19:28:18.396060 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:18.396019    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b0a36afc-349e-4a8a-8f3c-1694b6dd3510" path="/var/lib/kubelet/pods/b0a36afc-349e-4a8a-8f3c-1694b6dd3510/volumes"
Oct 30 19:28:18.396577 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:18.396512    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce4dfb91-07f4-429c-9499-153a8773e8c7" path="/var/lib/kubelet/pods/ce4dfb91-07f4-429c-9499-153a8773e8c7/volumes"
Oct 30 19:28:20.794688 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:20.794637    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"]
Oct 30 19:28:20.795852 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:20.795461    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b" podUID="9da5d19b-9f91-4d25-ab28-ffffb3475660" containerName="maas-api" containerID="cri-o://6ee07715aea6c1ca5129bc54662f23542355b9ff010bc985be639c7a60a8ddf0" gracePeriod=30
Oct 30 19:28:21.456930 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.456902    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"
Oct 30 19:28:21.595779 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.595739    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9da5d19b-9f91-4d25-ab28-ffffb3475660-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da5d19b-9f91-4d25-ab28-ffffb3475660-metrics-certs\") pod \"9da5d19b-9f91-4d25-ab28-ffffb3475660\" (UID: \"9da5d19b-9f91-4d25-ab28-ffffb3475660\") "
Oct 30 19:28:21.596050 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.596025    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9da5d19b-9f91-4d25-ab28-ffffb3475660-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9da5d19b-9f91-4d25-ab28-ffffb3475660-openshift-service-ca\") pod \"9da5d19b-9f91-4d25-ab28-ffffb3475660\" (UID: \"9da5d19b-9f91-4d25-ab28-ffffb3475660\") "
Oct 30 19:28:21.596690 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.596659    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9da5d19b-9f91-4d25-ab28-ffffb3475660-openshift-service-ca" pod "9da5d19b-9f91-4d25-ab28-ffffb3475660" (UID: "9da5d19b-9f91-4d25-ab28-ffffb3475660"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:28:21.596892 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.596872    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9da5d19b-9f91-4d25-ab28-ffffb3475660-kube-api-access-f8ght\" (UniqueName: \"kubernetes.io/projected/9da5d19b-9f91-4d25-ab28-ffffb3475660-kube-api-access-f8ght\") pod \"9da5d19b-9f91-4d25-ab28-ffffb3475660\" (UID: \"9da5d19b-9f91-4d25-ab28-ffffb3475660\") "
Oct 30 19:28:21.597460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.597005    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9da5d19b-9f91-4d25-ab28-ffffb3475660-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9da5d19b-9f91-4d25-ab28-ffffb3475660-maas-api-tls\") pod \"9da5d19b-9f91-4d25-ab28-ffffb3475660\" (UID: \"9da5d19b-9f91-4d25-ab28-ffffb3475660\") "
Oct 30 19:28:21.598256 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.598235    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9da5d19b-9f91-4d25-ab28-ffffb3475660-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:21.598817 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.598756    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9da5d19b-9f91-4d25-ab28-ffffb3475660-metrics-certs" pod "9da5d19b-9f91-4d25-ab28-ffffb3475660" (UID: "9da5d19b-9f91-4d25-ab28-ffffb3475660"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:28:21.604195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.604155    4524 generic.go:356] "Generic (PLEG): container finished" podID="9da5d19b-9f91-4d25-ab28-ffffb3475660" containerID="6ee07715aea6c1ca5129bc54662f23542355b9ff010bc985be639c7a60a8ddf0" exitCode=0
Oct 30 19:28:21.604302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.604217    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"
Oct 30 19:28:21.604412 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.604385    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b" event={"ID":"9da5d19b-9f91-4d25-ab28-ffffb3475660","Type":"ContainerDied","Data":"6ee07715aea6c1ca5129bc54662f23542355b9ff010bc985be639c7a60a8ddf0"}
Oct 30 19:28:21.604552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.604537    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b" event={"ID":"9da5d19b-9f91-4d25-ab28-ffffb3475660","Type":"ContainerDied","Data":"e85ece51f9da5ce2ff41e685832f9b9b86bdfbad4ba39b65207445001834bf6b"}
Oct 30 19:28:21.604676 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.604468    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9da5d19b-9f91-4d25-ab28-ffffb3475660-maas-api-tls" pod "9da5d19b-9f91-4d25-ab28-ffffb3475660" (UID: "9da5d19b-9f91-4d25-ab28-ffffb3475660"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:28:21.604676 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.604590    4524 scope.go:122] "RemoveContainer" containerID="6ee07715aea6c1ca5129bc54662f23542355b9ff010bc985be639c7a60a8ddf0"
Oct 30 19:28:21.611884 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.611847    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9da5d19b-9f91-4d25-ab28-ffffb3475660-kube-api-access-f8ght" pod "9da5d19b-9f91-4d25-ab28-ffffb3475660" (UID: "9da5d19b-9f91-4d25-ab28-ffffb3475660"). InnerVolumeSpecName "kube-api-access-f8ght". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:28:21.644049 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.644019    4524 scope.go:122] "RemoveContainer" containerID="6ee07715aea6c1ca5129bc54662f23542355b9ff010bc985be639c7a60a8ddf0"
Oct 30 19:28:21.644588 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:28:21.644564    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ee07715aea6c1ca5129bc54662f23542355b9ff010bc985be639c7a60a8ddf0\": container with ID starting with 6ee07715aea6c1ca5129bc54662f23542355b9ff010bc985be639c7a60a8ddf0 not found: ID does not exist" containerID="6ee07715aea6c1ca5129bc54662f23542355b9ff010bc985be639c7a60a8ddf0"
Oct 30 19:28:21.644675 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.644596    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ee07715aea6c1ca5129bc54662f23542355b9ff010bc985be639c7a60a8ddf0"} err="failed to get container status \"6ee07715aea6c1ca5129bc54662f23542355b9ff010bc985be639c7a60a8ddf0\": rpc error: code = NotFound desc = could not find container \"6ee07715aea6c1ca5129bc54662f23542355b9ff010bc985be639c7a60a8ddf0\": container with ID starting with 6ee07715aea6c1ca5129bc54662f23542355b9ff010bc985be639c7a60a8ddf0 not found: ID does not exist"
Oct 30 19:28:21.706951 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.704759    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9da5d19b-9f91-4d25-ab28-ffffb3475660-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:21.706951 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.704798    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da5d19b-9f91-4d25-ab28-ffffb3475660-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:21.706951 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.704816    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8ght\" (UniqueName: \"kubernetes.io/projected/9da5d19b-9f91-4d25-ab28-ffffb3475660-kube-api-access-f8ght\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:21.943133 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.943098    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"]
Oct 30 19:28:21.948777 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:21.948751    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0d360546-847b4f977c-s4p8b"]
Oct 30 19:28:22.559544 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:22.559501    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9da5d19b-9f91-4d25-ab28-ffffb3475660" path="/var/lib/kubelet/pods/9da5d19b-9f91-4d25-ab28-ffffb3475660/volumes"
Oct 30 19:28:29.929590 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:29.929543    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"]
Oct 30 19:28:29.931821 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:29.930919    4524 cpu_manager.go:414] "containerMap: removing container" podUID="ce4dfb91-07f4-429c-9499-153a8773e8c7" containerName="payload-pre-processing"
Oct 30 19:28:29.931821 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:29.930955    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce4dfb91-07f4-429c-9499-153a8773e8c7" containerName="payload-pre-processing"
Oct 30 19:28:29.941709 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:29.941674    4524 cpu_manager.go:414] "containerMap: removing container" podUID="b0a36afc-349e-4a8a-8f3c-1694b6dd3510" containerName="payload-processing"
Oct 30 19:28:29.941709 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:29.941702    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0a36afc-349e-4a8a-8f3c-1694b6dd3510" containerName="payload-processing"
Oct 30 19:28:29.941945 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:29.941757    4524 cpu_manager.go:414] "containerMap: removing container" podUID="9da5d19b-9f91-4d25-ab28-ffffb3475660" containerName="maas-api"
Oct 30 19:28:29.941945 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:29.941763    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9da5d19b-9f91-4d25-ab28-ffffb3475660" containerName="maas-api"
Oct 30 19:28:29.942061 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:29.941956    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="b0a36afc-349e-4a8a-8f3c-1694b6dd3510" containerName="payload-processing"
Oct 30 19:28:29.942061 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:29.941972    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="9da5d19b-9f91-4d25-ab28-ffffb3475660" containerName="maas-api"
Oct 30 19:28:29.942061 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:29.941985    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce4dfb91-07f4-429c-9499-153a8773e8c7" containerName="payload-pre-processing"
Oct 30 19:28:29.947848 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:29.947649    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"
Oct 30 19:28:29.959402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:29.956715    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"]
Oct 30 19:28:29.959402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:29.957964    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-8ceda0-serving-cert\""
Oct 30 19:28:30.098194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:30.093484    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08ffae59-b193-4e72-a5a2-ed99d33366f1-maas-api-tls\") pod \"maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv\" (UID: \"08ffae59-b193-4e72-a5a2-ed99d33366f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"
Oct 30 19:28:30.098194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:30.093584    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08ffae59-b193-4e72-a5a2-ed99d33366f1-openshift-service-ca\") pod \"maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv\" (UID: \"08ffae59-b193-4e72-a5a2-ed99d33366f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"
Oct 30 19:28:30.098194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:30.093633    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2gvl\" (UniqueName: \"kubernetes.io/projected/08ffae59-b193-4e72-a5a2-ed99d33366f1-kube-api-access-c2gvl\") pod \"maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv\" (UID: \"08ffae59-b193-4e72-a5a2-ed99d33366f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"
Oct 30 19:28:30.098194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:30.093989    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08ffae59-b193-4e72-a5a2-ed99d33366f1-metrics-certs\") pod \"maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv\" (UID: \"08ffae59-b193-4e72-a5a2-ed99d33366f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"
Oct 30 19:28:30.200792 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:30.200711    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08ffae59-b193-4e72-a5a2-ed99d33366f1-metrics-certs\") pod \"maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv\" (UID: \"08ffae59-b193-4e72-a5a2-ed99d33366f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"
Oct 30 19:28:30.201681 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:30.201654    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08ffae59-b193-4e72-a5a2-ed99d33366f1-maas-api-tls\") pod \"maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv\" (UID: \"08ffae59-b193-4e72-a5a2-ed99d33366f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"
Oct 30 19:28:30.202236 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:30.202217    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08ffae59-b193-4e72-a5a2-ed99d33366f1-openshift-service-ca\") pod \"maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv\" (UID: \"08ffae59-b193-4e72-a5a2-ed99d33366f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"
Oct 30 19:28:30.203065 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:30.203048    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2gvl\" (UniqueName: \"kubernetes.io/projected/08ffae59-b193-4e72-a5a2-ed99d33366f1-kube-api-access-c2gvl\") pod \"maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv\" (UID: \"08ffae59-b193-4e72-a5a2-ed99d33366f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"
Oct 30 19:28:30.203835 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:30.202972    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08ffae59-b193-4e72-a5a2-ed99d33366f1-openshift-service-ca\") pod \"maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv\" (UID: \"08ffae59-b193-4e72-a5a2-ed99d33366f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"
Oct 30 19:28:30.234193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:30.221439    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08ffae59-b193-4e72-a5a2-ed99d33366f1-maas-api-tls\") pod \"maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv\" (UID: \"08ffae59-b193-4e72-a5a2-ed99d33366f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"
Oct 30 19:28:30.303160 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:30.282817    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08ffae59-b193-4e72-a5a2-ed99d33366f1-metrics-certs\") pod \"maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv\" (UID: \"08ffae59-b193-4e72-a5a2-ed99d33366f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"
Oct 30 19:28:30.309136 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:30.309098    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2gvl\" (UniqueName: \"kubernetes.io/projected/08ffae59-b193-4e72-a5a2-ed99d33366f1-kube-api-access-c2gvl\") pod \"maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv\" (UID: \"08ffae59-b193-4e72-a5a2-ed99d33366f1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"
Oct 30 19:28:30.332297 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:30.332256    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"
Oct 30 19:28:30.909711 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:30.909622    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"]
Oct 30 19:28:30.936359 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:30.936264    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv" event={"ID":"08ffae59-b193-4e72-a5a2-ed99d33366f1","Type":"ContainerStarted","Data":"e0060aeaae2893628a6ee012457e5bb1fa01990be9f5de10da5c87816538591a"}
Oct 30 19:28:31.947185 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:31.947140    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv" event={"ID":"08ffae59-b193-4e72-a5a2-ed99d33366f1","Type":"ContainerStarted","Data":"ba655ea2981c26899f87928104654c1df27dfea810638928939a5308cf12a7dd"}
Oct 30 19:28:31.947616 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:31.947257    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"
Oct 30 19:28:31.969750 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:31.969689    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv" podStartSLOduration=2.509409456 podStartE2EDuration="2.969674326s" totalImagesPullingTime="460.26487ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:31.9677359 +0000 UTC m=+1343.708726970" watchObservedRunningTime="2026-10-30 19:28:31.969674326 +0000 UTC m=+1343.710665405"
Oct 30 19:28:37.960050 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:37.960019    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"
Oct 30 19:28:38.138321 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:38.138282    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"]
Oct 30 19:28:38.146824 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:38.146797    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"
Oct 30 19:28:38.154195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:38.151162    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-f4012a-serving-cert\""
Oct 30 19:28:38.154195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:38.153560    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"]
Oct 30 19:28:38.269254 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:38.265390    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ddef5788-e8d7-4f77-9828-a5557e3cc615-maas-api-tls\") pod \"maas-api-e2e-api-a-f4012a-966bbc888-pl46b\" (UID: \"ddef5788-e8d7-4f77-9828-a5557e3cc615\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"
Oct 30 19:28:38.269254 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:38.265443    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddef5788-e8d7-4f77-9828-a5557e3cc615-metrics-certs\") pod \"maas-api-e2e-api-a-f4012a-966bbc888-pl46b\" (UID: \"ddef5788-e8d7-4f77-9828-a5557e3cc615\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"
Oct 30 19:28:38.269254 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:38.265726    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnkt5\" (UniqueName: \"kubernetes.io/projected/ddef5788-e8d7-4f77-9828-a5557e3cc615-kube-api-access-cnkt5\") pod \"maas-api-e2e-api-a-f4012a-966bbc888-pl46b\" (UID: \"ddef5788-e8d7-4f77-9828-a5557e3cc615\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"
Oct 30 19:28:38.269254 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:38.265815    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddef5788-e8d7-4f77-9828-a5557e3cc615-openshift-service-ca\") pod \"maas-api-e2e-api-a-f4012a-966bbc888-pl46b\" (UID: \"ddef5788-e8d7-4f77-9828-a5557e3cc615\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"
Oct 30 19:28:38.372251 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:38.367429    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnkt5\" (UniqueName: \"kubernetes.io/projected/ddef5788-e8d7-4f77-9828-a5557e3cc615-kube-api-access-cnkt5\") pod \"maas-api-e2e-api-a-f4012a-966bbc888-pl46b\" (UID: \"ddef5788-e8d7-4f77-9828-a5557e3cc615\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"
Oct 30 19:28:38.372251 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:38.367540    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddef5788-e8d7-4f77-9828-a5557e3cc615-openshift-service-ca\") pod \"maas-api-e2e-api-a-f4012a-966bbc888-pl46b\" (UID: \"ddef5788-e8d7-4f77-9828-a5557e3cc615\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"
Oct 30 19:28:38.372251 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:38.367709    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ddef5788-e8d7-4f77-9828-a5557e3cc615-maas-api-tls\") pod \"maas-api-e2e-api-a-f4012a-966bbc888-pl46b\" (UID: \"ddef5788-e8d7-4f77-9828-a5557e3cc615\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"
Oct 30 19:28:38.372251 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:38.367736    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddef5788-e8d7-4f77-9828-a5557e3cc615-metrics-certs\") pod \"maas-api-e2e-api-a-f4012a-966bbc888-pl46b\" (UID: \"ddef5788-e8d7-4f77-9828-a5557e3cc615\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"
Oct 30 19:28:38.373875 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:38.373846    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddef5788-e8d7-4f77-9828-a5557e3cc615-openshift-service-ca\") pod \"maas-api-e2e-api-a-f4012a-966bbc888-pl46b\" (UID: \"ddef5788-e8d7-4f77-9828-a5557e3cc615\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"
Oct 30 19:28:38.374133 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:28:38.374117    4524 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-serving-cert: secret "maas-api-e2e-api-a-f4012a-serving-cert" not found
Oct 30 19:28:38.374300 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:28:38.374288    4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddef5788-e8d7-4f77-9828-a5557e3cc615-maas-api-tls podName:ddef5788-e8d7-4f77-9828-a5557e3cc615 nodeName:}" failed. No retries permitted until 2026-10-30 19:28:38.874267297 +0000 UTC m=+1350.615258354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ddef5788-e8d7-4f77-9828-a5557e3cc615-maas-api-tls") pod "maas-api-e2e-api-a-f4012a-966bbc888-pl46b" (UID: "ddef5788-e8d7-4f77-9828-a5557e3cc615") : secret "maas-api-e2e-api-a-f4012a-serving-cert" not found
Oct 30 19:28:38.411291 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:38.411261    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddef5788-e8d7-4f77-9828-a5557e3cc615-metrics-certs\") pod \"maas-api-e2e-api-a-f4012a-966bbc888-pl46b\" (UID: \"ddef5788-e8d7-4f77-9828-a5557e3cc615\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"
Oct 30 19:28:38.429914 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:38.429849    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnkt5\" (UniqueName: \"kubernetes.io/projected/ddef5788-e8d7-4f77-9828-a5557e3cc615-kube-api-access-cnkt5\") pod \"maas-api-e2e-api-a-f4012a-966bbc888-pl46b\" (UID: \"ddef5788-e8d7-4f77-9828-a5557e3cc615\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"
Oct 30 19:28:38.887625 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:38.887592    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ddef5788-e8d7-4f77-9828-a5557e3cc615-maas-api-tls\") pod \"maas-api-e2e-api-a-f4012a-966bbc888-pl46b\" (UID: \"ddef5788-e8d7-4f77-9828-a5557e3cc615\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"
Oct 30 19:28:38.890031 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:38.890002    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ddef5788-e8d7-4f77-9828-a5557e3cc615-maas-api-tls\") pod \"maas-api-e2e-api-a-f4012a-966bbc888-pl46b\" (UID: \"ddef5788-e8d7-4f77-9828-a5557e3cc615\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"
Oct 30 19:28:39.075987 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:39.075941    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"
Oct 30 19:28:39.624230 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:28:39.624197    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddef5788_e8d7_4f77_9828_a5557e3cc615.slice/crio-de1d0993976aaeab3c6d4d1f18ae79340abad4c1d29304e5d1469789a21a28ce WatchSource:0}: Error finding container de1d0993976aaeab3c6d4d1f18ae79340abad4c1d29304e5d1469789a21a28ce: Status 404 returned error can't find the container with id de1d0993976aaeab3c6d4d1f18ae79340abad4c1d29304e5d1469789a21a28ce
Oct 30 19:28:39.627223 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:39.627109    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"]
Oct 30 19:28:40.085657 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:40.085619    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b" event={"ID":"ddef5788-e8d7-4f77-9828-a5557e3cc615","Type":"ContainerStarted","Data":"de1d0993976aaeab3c6d4d1f18ae79340abad4c1d29304e5d1469789a21a28ce"}
Oct 30 19:28:41.103304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:41.103262    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b" event={"ID":"ddef5788-e8d7-4f77-9828-a5557e3cc615","Type":"ContainerStarted","Data":"053dcc1fc902afc430f9c5b025782a49329022caa5a686ca9443138653ad0daa"}
Oct 30 19:28:41.103807 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:41.103438    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"
Oct 30 19:28:45.668329 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:28:45.668287    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:28:47.117788 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:47.117757    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"
Oct 30 19:28:47.143980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:28:47.143927    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b" podStartSLOduration=8.688523782 podStartE2EDuration="9.1439144s" totalImagesPullingTime="455.390618ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:41.124417918 +0000 UTC m=+1352.865408996" watchObservedRunningTime="2026-10-30 19:28:47.1439144 +0000 UTC m=+1358.884905478"
Oct 30 19:29:15.721193 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:29:15.709727    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:29:21.244125 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:21.244046    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29"]
Oct 30 19:29:21.250204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:21.250180    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29"
Oct 30 19:29:21.264472 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:21.261897    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29"]
Oct 30 19:29:21.279540 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:21.279511    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmhcs\" (UniqueName: \"kubernetes.io/projected/0662c903-51a8-4e05-a90d-f3baf5488511-kube-api-access-dmhcs\") pod \"payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29\" (UID: \"0662c903-51a8-4e05-a90d-f3baf5488511\") " pod="openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29"
Oct 30 19:29:21.279692 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:21.279555    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0662c903-51a8-4e05-a90d-f3baf5488511-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29\" (UID: \"0662c903-51a8-4e05-a90d-f3baf5488511\") " pod="openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29"
Oct 30 19:29:21.333234 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:21.333196    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-afdf16\""
Oct 30 19:29:21.333632 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:21.333613    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-afdf16-dockercfg-9lt4p\""
Oct 30 19:29:21.416419 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:21.387571    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmhcs\" (UniqueName: \"kubernetes.io/projected/0662c903-51a8-4e05-a90d-f3baf5488511-kube-api-access-dmhcs\") pod \"payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29\" (UID: \"0662c903-51a8-4e05-a90d-f3baf5488511\") " pod="openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29"
Oct 30 19:29:21.416762 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:21.416727    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0662c903-51a8-4e05-a90d-f3baf5488511-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29\" (UID: \"0662c903-51a8-4e05-a90d-f3baf5488511\") " pod="openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29"
Oct 30 19:29:21.417987 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:21.417959    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0662c903-51a8-4e05-a90d-f3baf5488511-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29\" (UID: \"0662c903-51a8-4e05-a90d-f3baf5488511\") " pod="openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29"
Oct 30 19:29:21.531235 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:21.530661    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmhcs\" (UniqueName: \"kubernetes.io/projected/0662c903-51a8-4e05-a90d-f3baf5488511-kube-api-access-dmhcs\") pod \"payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29\" (UID: \"0662c903-51a8-4e05-a90d-f3baf5488511\") " pod="openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29"
Oct 30 19:29:21.584198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:21.582200    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29"
Oct 30 19:29:22.131969 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:22.131939    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29"]
Oct 30 19:29:22.133645 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:29:22.133611    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0662c903_51a8_4e05_a90d_f3baf5488511.slice/crio-f9aa17f285b9758014e59ad27c1288a7311e866dd612f81cac39be10db823ae7 WatchSource:0}: Error finding container f9aa17f285b9758014e59ad27c1288a7311e866dd612f81cac39be10db823ae7: Status 404 returned error can't find the container with id f9aa17f285b9758014e59ad27c1288a7311e866dd612f81cac39be10db823ae7
Oct 30 19:29:22.532031 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:22.531945    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29" event={"ID":"0662c903-51a8-4e05-a90d-f3baf5488511","Type":"ContainerStarted","Data":"69f864fbf3d4bfecde11a3523fa3d60c50e38888bc29cf06b2c101c4a9cb1152"}
Oct 30 19:29:22.532031 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:22.532001    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29" event={"ID":"0662c903-51a8-4e05-a90d-f3baf5488511","Type":"ContainerStarted","Data":"f9aa17f285b9758014e59ad27c1288a7311e866dd612f81cac39be10db823ae7"}
Oct 30 19:29:22.532589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:22.532135    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29"
Oct 30 19:29:22.551476 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:22.551320    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29" podStartSLOduration=1.551303634 podStartE2EDuration="1.551303634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:22.550665031 +0000 UTC m=+1394.291656111" watchObservedRunningTime="2026-10-30 19:29:22.551303634 +0000 UTC m=+1394.292294716"
Oct 30 19:29:26.203193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:26.200870    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz" podUID="e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:29:26.393220 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:26.314255    4524 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j" podUID="24fbbf1e-3438-4436-b4cd-9890fe28676a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.98:8000/health\": context deadline exceeded"
Oct 30 19:29:33.548828 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:33.548798    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29"
Oct 30 19:29:44.919887 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:44.919846    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"]
Oct 30 19:29:44.921457 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:44.921139    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p" podUID="299c6068-7c79-4610-bfe2-894dd531c068" containerName="maas-api" containerID="cri-o://d960fd013cc3b29f47f06d969ce0bee60c6ca2d36cd497041414b0b3352243c1" gracePeriod=30
Oct 30 19:29:45.816980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.816730    4524 generic.go:356] "Generic (PLEG): container finished" podID="299c6068-7c79-4610-bfe2-894dd531c068" containerID="d960fd013cc3b29f47f06d969ce0bee60c6ca2d36cd497041414b0b3352243c1" exitCode=0
Oct 30 19:29:45.817742 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.817418    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p" event={"ID":"299c6068-7c79-4610-bfe2-894dd531c068","Type":"ContainerDied","Data":"d960fd013cc3b29f47f06d969ce0bee60c6ca2d36cd497041414b0b3352243c1"}
Oct 30 19:29:45.855327 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:29:45.855292    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:29:45.880892 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.880863    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"
Oct 30 19:29:45.913305 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.913277    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"]
Oct 30 19:29:45.914145 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.914128    4524 cpu_manager.go:414] "containerMap: removing container" podUID="299c6068-7c79-4610-bfe2-894dd531c068" containerName="maas-api"
Oct 30 19:29:45.914252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.914148    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="299c6068-7c79-4610-bfe2-894dd531c068" containerName="maas-api"
Oct 30 19:29:45.914297 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.914280    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="299c6068-7c79-4610-bfe2-894dd531c068" containerName="maas-api"
Oct 30 19:29:45.918292 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.918256    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"
Oct 30 19:29:45.929990 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.929968    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"]
Oct 30 19:29:45.970742 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.970705    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-afdf16-serving-cert\""
Oct 30 19:29:45.995078 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.995050    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/299c6068-7c79-4610-bfe2-894dd531c068-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/299c6068-7c79-4610-bfe2-894dd531c068-maas-api-tls\") pod \"299c6068-7c79-4610-bfe2-894dd531c068\" (UID: \"299c6068-7c79-4610-bfe2-894dd531c068\") "
Oct 30 19:29:45.995265 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.995248    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/299c6068-7c79-4610-bfe2-894dd531c068-kube-api-access-thtpw\" (UniqueName: \"kubernetes.io/projected/299c6068-7c79-4610-bfe2-894dd531c068-kube-api-access-thtpw\") pod \"299c6068-7c79-4610-bfe2-894dd531c068\" (UID: \"299c6068-7c79-4610-bfe2-894dd531c068\") "
Oct 30 19:29:45.995333 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.995292    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/299c6068-7c79-4610-bfe2-894dd531c068-metrics-certs\" (UniqueName: \"kubernetes.io/secret/299c6068-7c79-4610-bfe2-894dd531c068-metrics-certs\") pod \"299c6068-7c79-4610-bfe2-894dd531c068\" (UID: \"299c6068-7c79-4610-bfe2-894dd531c068\") "
Oct 30 19:29:45.995393 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.995332    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/299c6068-7c79-4610-bfe2-894dd531c068-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/299c6068-7c79-4610-bfe2-894dd531c068-openshift-service-ca\") pod \"299c6068-7c79-4610-bfe2-894dd531c068\" (UID: \"299c6068-7c79-4610-bfe2-894dd531c068\") "
Oct 30 19:29:45.996081 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.996047    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/299c6068-7c79-4610-bfe2-894dd531c068-openshift-service-ca" pod "299c6068-7c79-4610-bfe2-894dd531c068" (UID: "299c6068-7c79-4610-bfe2-894dd531c068"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:29:45.996677 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.996647    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-metrics-certs\") pod \"maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm\" (UID: \"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"
Oct 30 19:29:45.996796 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.996763    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-maas-api-tls\") pod \"maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm\" (UID: \"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"
Oct 30 19:29:45.996994 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.996961    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm\" (UID: \"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"
Oct 30 19:29:45.997110 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.997044    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lsmz\" (UniqueName: \"kubernetes.io/projected/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-kube-api-access-8lsmz\") pod \"maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm\" (UID: \"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"
Oct 30 19:29:45.997501 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.997482    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/299c6068-7c79-4610-bfe2-894dd531c068-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:45.997845 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.997824    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/299c6068-7c79-4610-bfe2-894dd531c068-maas-api-tls" pod "299c6068-7c79-4610-bfe2-894dd531c068" (UID: "299c6068-7c79-4610-bfe2-894dd531c068"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:29:45.999216 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.998766    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/299c6068-7c79-4610-bfe2-894dd531c068-kube-api-access-thtpw" pod "299c6068-7c79-4610-bfe2-894dd531c068" (UID: "299c6068-7c79-4610-bfe2-894dd531c068"). InnerVolumeSpecName "kube-api-access-thtpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:29:45.999554 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:45.999531    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/299c6068-7c79-4610-bfe2-894dd531c068-metrics-certs" pod "299c6068-7c79-4610-bfe2-894dd531c068" (UID: "299c6068-7c79-4610-bfe2-894dd531c068"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:29:46.111195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:46.102287    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-metrics-certs\") pod \"maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm\" (UID: \"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"
Oct 30 19:29:46.111195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:46.102360    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-maas-api-tls\") pod \"maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm\" (UID: \"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"
Oct 30 19:29:46.111195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:46.102459    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm\" (UID: \"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"
Oct 30 19:29:46.111195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:46.102522    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lsmz\" (UniqueName: \"kubernetes.io/projected/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-kube-api-access-8lsmz\") pod \"maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm\" (UID: \"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"
Oct 30 19:29:46.111195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:46.102721    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/299c6068-7c79-4610-bfe2-894dd531c068-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:46.111195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:46.102740    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thtpw\" (UniqueName: \"kubernetes.io/projected/299c6068-7c79-4610-bfe2-894dd531c068-kube-api-access-thtpw\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:46.111195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:46.102755    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/299c6068-7c79-4610-bfe2-894dd531c068-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:46.124201 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:46.113968    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm\" (UID: \"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"
Oct 30 19:29:46.139191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:46.136788    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-metrics-certs\") pod \"maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm\" (UID: \"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"
Oct 30 19:29:46.150974 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:46.149832    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-maas-api-tls\") pod \"maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm\" (UID: \"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"
Oct 30 19:29:46.177325 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:46.175064    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lsmz\" (UniqueName: \"kubernetes.io/projected/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-kube-api-access-8lsmz\") pod \"maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm\" (UID: \"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"
Oct 30 19:29:46.256962 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:46.256922    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"
Oct 30 19:29:46.657666 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:29:46.657606    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4772e6be_0a1a_4def_b2fa_c4f09f7df3a3.slice/crio-319e1888be437c45d46a84a981cc9fb9ad1347928503c2214b897dbb94ee17ac WatchSource:0}: Error finding container 319e1888be437c45d46a84a981cc9fb9ad1347928503c2214b897dbb94ee17ac: Status 404 returned error can't find the container with id 319e1888be437c45d46a84a981cc9fb9ad1347928503c2214b897dbb94ee17ac
Oct 30 19:29:46.657847 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:46.657825    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"]
Oct 30 19:29:46.659011 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:46.658997    4524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:29:46.830161 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:46.830122    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm" event={"ID":"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3","Type":"ContainerStarted","Data":"319e1888be437c45d46a84a981cc9fb9ad1347928503c2214b897dbb94ee17ac"}
Oct 30 19:29:46.831953 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:46.831926    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"
Oct 30 19:29:46.832071 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:46.831966    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p" event={"ID":"299c6068-7c79-4610-bfe2-894dd531c068","Type":"ContainerDied","Data":"3e9c37122f0a0a7f79806ce6702237d6590876f3979d30324bf146067db801c0"}
Oct 30 19:29:46.832071 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:46.832029    4524 scope.go:122] "RemoveContainer" containerID="d960fd013cc3b29f47f06d969ce0bee60c6ca2d36cd497041414b0b3352243c1"
Oct 30 19:29:46.870383 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:46.870348    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"]
Oct 30 19:29:46.874951 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:46.874925    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-9b2e27-b9dd88f66-8p95p"]
Oct 30 19:29:47.842884 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:47.842841    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm" event={"ID":"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3","Type":"ContainerStarted","Data":"c8a1662c12c34bc51ef7769c12adac05796d87a567b0c72d4fa46f02224c5d6c"}
Oct 30 19:29:47.843385 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:47.842936    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"
Oct 30 19:29:47.866496 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:47.866443    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm" podStartSLOduration=8.43381664 podStartE2EDuration="8.866429011s" totalImagesPullingTime="432.612371ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:47.861711699 +0000 UTC m=+1419.602702788" watchObservedRunningTime="2026-10-30 19:29:47.866429011 +0000 UTC m=+1419.607420089"
Oct 30 19:29:48.455142 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:48.455102    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="299c6068-7c79-4610-bfe2-894dd531c068" path="/var/lib/kubelet/pods/299c6068-7c79-4610-bfe2-894dd531c068/volumes"
Oct 30 19:29:50.806728 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:50.806691    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"]
Oct 30 19:29:50.807316 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:50.806960    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6" podUID="75f42a71-342e-4c05-80c7-a1e7b6336984" containerName="istio-proxy" containerID="cri-o://113ea6419af0ef9d096c81a5337ee8d5571cea01cce84b280f9969ab08ad43fb" gracePeriod=30
Oct 30 19:29:53.858774 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:53.858741    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"
Oct 30 19:29:55.945754 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:55.945725    4524 generic.go:356] "Generic (PLEG): container finished" podID="75f42a71-342e-4c05-80c7-a1e7b6336984" containerID="113ea6419af0ef9d096c81a5337ee8d5571cea01cce84b280f9969ab08ad43fb" exitCode=0
Oct 30 19:29:55.946151 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:55.945799    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6" event={"ID":"75f42a71-342e-4c05-80c7-a1e7b6336984","Type":"ContainerDied","Data":"113ea6419af0ef9d096c81a5337ee8d5571cea01cce84b280f9969ab08ad43fb"}
Oct 30 19:29:56.232639 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.232612    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:29:56.271136 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.271104    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"]
Oct 30 19:29:56.272479 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.272451    4524 cpu_manager.go:414] "containerMap: removing container" podUID="75f42a71-342e-4c05-80c7-a1e7b6336984" containerName="istio-proxy"
Oct 30 19:29:56.272479 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.272477    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75f42a71-342e-4c05-80c7-a1e7b6336984" containerName="istio-proxy"
Oct 30 19:29:56.272719 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.272702    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="75f42a71-342e-4c05-80c7-a1e7b6336984" containerName="istio-proxy"
Oct 30 19:29:56.277313 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.277287    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.291826 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.291795    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"]
Oct 30 19:29:56.312549 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.312515    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-48b1a6a7-openshift-default-dockercfg-gwncx\""
Oct 30 19:29:56.314432 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.314401    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-workload-certs\") pod \"75f42a71-342e-4c05-80c7-a1e7b6336984\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") "
Oct 30 19:29:56.314558 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.314548    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-token\" (UniqueName: \"kubernetes.io/projected/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-token\") pod \"75f42a71-342e-4c05-80c7-a1e7b6336984\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") "
Oct 30 19:29:56.314632 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.314615    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-credential-socket\") pod \"75f42a71-342e-4c05-80c7-a1e7b6336984\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") "
Oct 30 19:29:56.314751 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.314735    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-workload-socket\") pod \"75f42a71-342e-4c05-80c7-a1e7b6336984\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") "
Oct 30 19:29:56.314816 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.314802    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-podinfo\") pod \"75f42a71-342e-4c05-80c7-a1e7b6336984\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") "
Oct 30 19:29:56.314907 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.314892    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75f42a71-342e-4c05-80c7-a1e7b6336984-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75f42a71-342e-4c05-80c7-a1e7b6336984-istiod-ca-cert\") pod \"75f42a71-342e-4c05-80c7-a1e7b6336984\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") "
Oct 30 19:29:56.314966 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.314926    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75f42a71-342e-4c05-80c7-a1e7b6336984-kube-api-access-cqt42\" (UniqueName: \"kubernetes.io/projected/75f42a71-342e-4c05-80c7-a1e7b6336984-kube-api-access-cqt42\") pod \"75f42a71-342e-4c05-80c7-a1e7b6336984\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") "
Oct 30 19:29:56.315025 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.314971    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-envoy\") pod \"75f42a71-342e-4c05-80c7-a1e7b6336984\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") "
Oct 30 19:29:56.315025 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.315004    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-data\") pod \"75f42a71-342e-4c05-80c7-a1e7b6336984\" (UID: \"75f42a71-342e-4c05-80c7-a1e7b6336984\") "
Oct 30 19:29:56.315267 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.315242    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-workload-certs\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.315364 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.315330    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-istio-data\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.315421 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.315368    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slgkl\" (UniqueName: \"kubernetes.io/projected/62577a04-5c57-4428-af59-56403ecb30fa-kube-api-access-slgkl\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.315421 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.315404    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62577a04-5c57-4428-af59-56403ecb30fa-istio-podinfo\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.315523 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.315435    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-workload-socket\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.315580 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.315518    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62577a04-5c57-4428-af59-56403ecb30fa-istiod-ca-cert\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.315636 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.315583    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-istio-envoy\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.315636 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.315614    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62577a04-5c57-4428-af59-56403ecb30fa-istio-token\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.315741 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.315663    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-credential-socket\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.319042 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.318897    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-data" pod "75f42a71-342e-4c05-80c7-a1e7b6336984" (UID: "75f42a71-342e-4c05-80c7-a1e7b6336984"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:29:56.319156 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.319085    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-credential-socket" pod "75f42a71-342e-4c05-80c7-a1e7b6336984" (UID: "75f42a71-342e-4c05-80c7-a1e7b6336984"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:29:56.319236 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.319185    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75f42a71-342e-4c05-80c7-a1e7b6336984-istiod-ca-cert" pod "75f42a71-342e-4c05-80c7-a1e7b6336984" (UID: "75f42a71-342e-4c05-80c7-a1e7b6336984"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:29:56.319466 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.319432    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-workload-socket" pod "75f42a71-342e-4c05-80c7-a1e7b6336984" (UID: "75f42a71-342e-4c05-80c7-a1e7b6336984"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:29:56.321736 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.321681    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-podinfo" pod "75f42a71-342e-4c05-80c7-a1e7b6336984" (UID: "75f42a71-342e-4c05-80c7-a1e7b6336984"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:29:56.321848 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.321731    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75f42a71-342e-4c05-80c7-a1e7b6336984-kube-api-access-cqt42" pod "75f42a71-342e-4c05-80c7-a1e7b6336984" (UID: "75f42a71-342e-4c05-80c7-a1e7b6336984"). InnerVolumeSpecName "kube-api-access-cqt42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:29:56.325788 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.325762    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-workload-certs" pod "75f42a71-342e-4c05-80c7-a1e7b6336984" (UID: "75f42a71-342e-4c05-80c7-a1e7b6336984"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:29:56.328143 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.328084    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-envoy" pod "75f42a71-342e-4c05-80c7-a1e7b6336984" (UID: "75f42a71-342e-4c05-80c7-a1e7b6336984"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:29:56.328452 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.328404    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-token" pod "75f42a71-342e-4c05-80c7-a1e7b6336984" (UID: "75f42a71-342e-4c05-80c7-a1e7b6336984"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:29:56.417781 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.417508    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62577a04-5c57-4428-af59-56403ecb30fa-istiod-ca-cert\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.417781 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.417574    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-istio-envoy\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.417781 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.417616    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62577a04-5c57-4428-af59-56403ecb30fa-istio-token\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.417781 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.417658    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-credential-socket\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.418037 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.417951    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-workload-certs\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.418096 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.418039    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-istio-data\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.418096 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.418079    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slgkl\" (UniqueName: \"kubernetes.io/projected/62577a04-5c57-4428-af59-56403ecb30fa-kube-api-access-slgkl\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.418252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.418118    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62577a04-5c57-4428-af59-56403ecb30fa-istio-podinfo\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.418252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.418243    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-workload-socket\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.418326 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.418291    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-credential-socket\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.418368 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.418329    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:56.418368 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.418348    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:56.418368 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.418363    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:56.418468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.418379    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75f42a71-342e-4c05-80c7-a1e7b6336984-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:56.418468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.418394    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqt42\" (UniqueName: \"kubernetes.io/projected/75f42a71-342e-4c05-80c7-a1e7b6336984-kube-api-access-cqt42\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:56.418468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.418410    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:56.418468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.418423    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:56.418468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.418437    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75f42a71-342e-4c05-80c7-a1e7b6336984-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:56.418468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.418451    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75f42a71-342e-4c05-80c7-a1e7b6336984-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:29:56.418715 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.418692    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62577a04-5c57-4428-af59-56403ecb30fa-istiod-ca-cert\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.418764 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.418725    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-workload-socket\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.418999 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.418977    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-workload-certs\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.419732 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.419443    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-istio-data\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.420870 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.420849    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-istio-envoy\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.421889 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.421869    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62577a04-5c57-4428-af59-56403ecb30fa-istio-podinfo\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.430760 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.430737    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62577a04-5c57-4428-af59-56403ecb30fa-istio-token\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.431434 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.431411    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slgkl\" (UniqueName: \"kubernetes.io/projected/62577a04-5c57-4428-af59-56403ecb30fa-kube-api-access-slgkl\") pod \"e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") " pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.598909 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.598880    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:56.973804 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.973756    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"]
Oct 30 19:29:56.997745 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.996920    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:29:56.997745 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.997011    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:29:56.997745 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.997054    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:29:57.000557 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.999254    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6" event={"ID":"75f42a71-342e-4c05-80c7-a1e7b6336984","Type":"ContainerDied","Data":"28c17fc54c8cbe7d17b8c22b8d10993bb1a36fd591e4198d2060b82f782eb792"}
Oct 30 19:29:57.000557 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.999318    4524 scope.go:122] "RemoveContainer" containerID="113ea6419af0ef9d096c81a5337ee8d5571cea01cce84b280f9969ab08ad43fb"
Oct 30 19:29:57.000557 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:56.999472    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"
Oct 30 19:29:57.035121 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:57.035096    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"]
Oct 30 19:29:57.040383 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:57.040358    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-ttfb6"]
Oct 30 19:29:58.034323 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:58.032432    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24" event={"ID":"62577a04-5c57-4428-af59-56403ecb30fa","Type":"ContainerStarted","Data":"22cb7997b22b713038f32a165d352526594fe90953cb8141a694e197b4892fb4"}
Oct 30 19:29:58.034323 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:58.032498    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24" event={"ID":"62577a04-5c57-4428-af59-56403ecb30fa","Type":"ContainerStarted","Data":"8478547abefacfba7e9d5ca7a11f49340a2f3c9ae707c56140be523ff683acc4"}
Oct 30 19:29:58.071287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:58.071227    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24" podStartSLOduration=4.071206937 podStartE2EDuration="4.071206937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:58.068150538 +0000 UTC m=+1429.809141618" watchObservedRunningTime="2026-10-30 19:29:58.071206937 +0000 UTC m=+1429.812198016"
Oct 30 19:29:58.409270 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:58.409237    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75f42a71-342e-4c05-80c7-a1e7b6336984" path="/var/lib/kubelet/pods/75f42a71-342e-4c05-80c7-a1e7b6336984/volumes"
Oct 30 19:29:58.600116 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:58.600083    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:58.696688 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:58.696605    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:59.043445 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:59.043361    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:29:59.044511 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:29:59.044481    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:30:00.305521 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.305491    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x"]
Oct 30 19:30:00.423472 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.423443    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x"
Oct 30 19:30:00.458542 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.458506    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z26xs\""
Oct 30 19:30:00.489552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.489506    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m29h9\" (UniqueName: \"kubernetes.io/projected/3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d-kube-api-access-m29h9\") pod \"maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x\" (UID: \"3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x"
Oct 30 19:30:00.489951 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.489929    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x\" (UID: \"3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x"
Oct 30 19:30:00.638840 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.638806    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt"]
Oct 30 19:30:00.642688 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.642657    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m29h9\" (UniqueName: \"kubernetes.io/projected/3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d-kube-api-access-m29h9\") pod \"maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x\" (UID: \"3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x"
Oct 30 19:30:00.643003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.642978    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x\" (UID: \"3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x"
Oct 30 19:30:00.645142 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.644249    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x\" (UID: \"3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x"
Oct 30 19:30:00.646474 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.646456    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt"
Oct 30 19:30:00.654503 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.654475    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp"]
Oct 30 19:30:00.661416 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.661395    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp"
Oct 30 19:30:00.678357 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.678330    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m29h9\" (UniqueName: \"kubernetes.io/projected/3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d-kube-api-access-m29h9\") pod \"maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x\" (UID: \"3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x"
Oct 30 19:30:00.681224 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.681200    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9"]
Oct 30 19:30:00.688238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.688156    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9"
Oct 30 19:30:00.713271 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.709116    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x"]
Oct 30 19:30:00.730509 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.726459    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt"]
Oct 30 19:30:00.730509 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.728393    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9"]
Oct 30 19:30:00.736279 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.736246    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp"]
Oct 30 19:30:00.745061 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.745028    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mvxh\" (UniqueName: \"kubernetes.io/projected/c436fb87-4351-4a27-9286-216883df85cd-kube-api-access-6mvxh\") pod \"maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt\" (UID: \"c436fb87-4351-4a27-9286-216883df85cd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt"
Oct 30 19:30:00.745220 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.745089    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfscb\" (UniqueName: \"kubernetes.io/projected/7543e764-a757-4962-8342-f6d9d9e0f368-kube-api-access-dfscb\") pod \"maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp\" (UID: \"7543e764-a757-4962-8342-f6d9d9e0f368\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp"
Oct 30 19:30:00.745298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.745218    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7543e764-a757-4962-8342-f6d9d9e0f368-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp\" (UID: \"7543e764-a757-4962-8342-f6d9d9e0f368\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp"
Oct 30 19:30:00.745298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.745269    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c436fb87-4351-4a27-9286-216883df85cd-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt\" (UID: \"c436fb87-4351-4a27-9286-216883df85cd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt"
Oct 30 19:30:00.853544 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.853507    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w4sv\" (UniqueName: \"kubernetes.io/projected/652b5102-d21a-4673-b955-fb005179323e-kube-api-access-9w4sv\") pod \"maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9\" (UID: \"652b5102-d21a-4673-b955-fb005179323e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9"
Oct 30 19:30:00.853741 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.853596    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mvxh\" (UniqueName: \"kubernetes.io/projected/c436fb87-4351-4a27-9286-216883df85cd-kube-api-access-6mvxh\") pod \"maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt\" (UID: \"c436fb87-4351-4a27-9286-216883df85cd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt"
Oct 30 19:30:00.853741 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.853699    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfscb\" (UniqueName: \"kubernetes.io/projected/7543e764-a757-4962-8342-f6d9d9e0f368-kube-api-access-dfscb\") pod \"maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp\" (UID: \"7543e764-a757-4962-8342-f6d9d9e0f368\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp"
Oct 30 19:30:00.854156 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.854136    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7543e764-a757-4962-8342-f6d9d9e0f368-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp\" (UID: \"7543e764-a757-4962-8342-f6d9d9e0f368\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp"
Oct 30 19:30:00.854246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.854206    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c436fb87-4351-4a27-9286-216883df85cd-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt\" (UID: \"c436fb87-4351-4a27-9286-216883df85cd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt"
Oct 30 19:30:00.854314 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.854280    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/652b5102-d21a-4673-b955-fb005179323e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9\" (UID: \"652b5102-d21a-4673-b955-fb005179323e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9"
Oct 30 19:30:00.854981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.854960    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7543e764-a757-4962-8342-f6d9d9e0f368-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp\" (UID: \"7543e764-a757-4962-8342-f6d9d9e0f368\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp"
Oct 30 19:30:00.856354 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.856303    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c436fb87-4351-4a27-9286-216883df85cd-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt\" (UID: \"c436fb87-4351-4a27-9286-216883df85cd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt"
Oct 30 19:30:00.876589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.876467    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfscb\" (UniqueName: \"kubernetes.io/projected/7543e764-a757-4962-8342-f6d9d9e0f368-kube-api-access-dfscb\") pod \"maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp\" (UID: \"7543e764-a757-4962-8342-f6d9d9e0f368\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp"
Oct 30 19:30:00.891323 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.891153    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mvxh\" (UniqueName: \"kubernetes.io/projected/c436fb87-4351-4a27-9286-216883df85cd-kube-api-access-6mvxh\") pod \"maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt\" (UID: \"c436fb87-4351-4a27-9286-216883df85cd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt"
Oct 30 19:30:00.919025 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.918683    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x"
Oct 30 19:30:00.956390 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.956360    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w4sv\" (UniqueName: \"kubernetes.io/projected/652b5102-d21a-4673-b955-fb005179323e-kube-api-access-9w4sv\") pod \"maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9\" (UID: \"652b5102-d21a-4673-b955-fb005179323e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9"
Oct 30 19:30:00.956877 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.956863    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/652b5102-d21a-4673-b955-fb005179323e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9\" (UID: \"652b5102-d21a-4673-b955-fb005179323e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9"
Oct 30 19:30:00.957683 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.957664    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/652b5102-d21a-4673-b955-fb005179323e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9\" (UID: \"652b5102-d21a-4673-b955-fb005179323e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9"
Oct 30 19:30:00.980807 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:00.980727    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w4sv\" (UniqueName: \"kubernetes.io/projected/652b5102-d21a-4673-b955-fb005179323e-kube-api-access-9w4sv\") pod \"maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9\" (UID: \"652b5102-d21a-4673-b955-fb005179323e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9"
Oct 30 19:30:01.034514 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:01.034482    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt"
Oct 30 19:30:01.049373 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:01.049131    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp"
Oct 30 19:30:01.061221 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:01.060428    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9"
Oct 30 19:30:01.262533 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:30:01.262470    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aebf7f1_3afe_4a1b_97c2_df22c5f8ec0d.slice/crio-ba045246497de8df2960670c61c9a12b839b6b401daefa805ba92358bffa6765 WatchSource:0}: Error finding container ba045246497de8df2960670c61c9a12b839b6b401daefa805ba92358bffa6765: Status 404 returned error can't find the container with id ba045246497de8df2960670c61c9a12b839b6b401daefa805ba92358bffa6765
Oct 30 19:30:01.263155 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:01.263128    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x"]
Oct 30 19:30:01.418830 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:01.416731    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp"]
Oct 30 19:30:01.420652 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:30:01.420624    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc436fb87_4351_4a27_9286_216883df85cd.slice/crio-4359afc8a5ee4a29e3310084c1401c771da423019700d04b003b072ab7c61643 WatchSource:0}: Error finding container 4359afc8a5ee4a29e3310084c1401c771da423019700d04b003b072ab7c61643: Status 404 returned error can't find the container with id 4359afc8a5ee4a29e3310084c1401c771da423019700d04b003b072ab7c61643
Oct 30 19:30:01.421065 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:01.421035    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt"]
Oct 30 19:30:01.433627 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:30:01.433583    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod652b5102_d21a_4673_b955_fb005179323e.slice/crio-de2d4b380a7fe739bdc3e74b7b246873f73d73020a61e6fab505327b5944f4fa WatchSource:0}: Error finding container de2d4b380a7fe739bdc3e74b7b246873f73d73020a61e6fab505327b5944f4fa: Status 404 returned error can't find the container with id de2d4b380a7fe739bdc3e74b7b246873f73d73020a61e6fab505327b5944f4fa
Oct 30 19:30:01.434915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:01.434892    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9"]
Oct 30 19:30:02.217226 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:02.206621    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9" event={"ID":"652b5102-d21a-4673-b955-fb005179323e","Type":"ContainerStarted","Data":"de2d4b380a7fe739bdc3e74b7b246873f73d73020a61e6fab505327b5944f4fa"}
Oct 30 19:30:02.250438 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:02.250394    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp" event={"ID":"7543e764-a757-4962-8342-f6d9d9e0f368","Type":"ContainerStarted","Data":"a259a0e9ca5f6db02faaa505b43d8da9a6267e57efea4ae083c72f5f515a815f"}
Oct 30 19:30:02.282338 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:02.282258    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x" event={"ID":"3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d","Type":"ContainerStarted","Data":"ba045246497de8df2960670c61c9a12b839b6b401daefa805ba92358bffa6765"}
Oct 30 19:30:02.288104 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:02.288060    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt" event={"ID":"c436fb87-4351-4a27-9286-216883df85cd","Type":"ContainerStarted","Data":"4359afc8a5ee4a29e3310084c1401c771da423019700d04b003b072ab7c61643"}
Oct 30 19:30:02.374767 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:02.374735    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"]
Oct 30 19:30:02.375613 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:02.375579    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24" podUID="62577a04-5c57-4428-af59-56403ecb30fa" containerName="istio-proxy" containerID="cri-o://22cb7997b22b713038f32a165d352526594fe90953cb8141a694e197b4892fb4" gracePeriod=30
Oct 30 19:30:05.783591 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:05.783537    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"]
Oct 30 19:30:05.785089 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:05.785016    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9" podUID="83f517e5-9b8e-425a-845b-56253704cd78" containerName="istio-proxy" containerID="cri-o://b71879018815d86e282d8f0e69001291a1091babd9d025219863a4f39a8df76d" gracePeriod=30
Oct 30 19:30:06.490126 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:06.489966    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"]
Oct 30 19:30:06.490732 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:06.490257    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n" podUID="555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" containerName="istio-proxy" containerID="cri-o://81d6076e7f972a1be08e4feaf13fb91095394764249adff8a8909f31913def37" gracePeriod=30
Oct 30 19:30:06.505133 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:06.505101    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"]
Oct 30 19:30:06.505451 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:06.505352    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk" podUID="a4fd9180-f4aa-4986-b2f0-955296fe17ee" containerName="istio-proxy" containerID="cri-o://da7324c6301810498df84c172f6661cc8934360a23252f40bc67b5bd2a8d9ce3" gracePeriod=30
Oct 30 19:30:07.886624 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:07.886592    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:30:07.945109 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:07.945075    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85"]
Oct 30 19:30:07.979277 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:07.948856    4524 cpu_manager.go:414] "containerMap: removing container" podUID="62577a04-5c57-4428-af59-56403ecb30fa" containerName="istio-proxy"
Oct 30 19:30:07.989193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:07.980259    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="62577a04-5c57-4428-af59-56403ecb30fa" containerName="istio-proxy"
Oct 30 19:30:07.989726 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:07.989698    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="62577a04-5c57-4428-af59-56403ecb30fa" containerName="istio-proxy"
Oct 30 19:30:08.009936 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:07.988419    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-istio-envoy" pod "62577a04-5c57-4428-af59-56403ecb30fa" (UID: "62577a04-5c57-4428-af59-56403ecb30fa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:08.010103 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:07.983413    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-istio-envoy\") pod \"62577a04-5c57-4428-af59-56403ecb30fa\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") "
Oct 30 19:30:08.010314 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.010293    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-workload-certs\") pod \"62577a04-5c57-4428-af59-56403ecb30fa\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") "
Oct 30 19:30:08.010479 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.010459    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/62577a04-5c57-4428-af59-56403ecb30fa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62577a04-5c57-4428-af59-56403ecb30fa-istio-podinfo\") pod \"62577a04-5c57-4428-af59-56403ecb30fa\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") "
Oct 30 19:30:08.010717 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.010699    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-workload-socket\") pod \"62577a04-5c57-4428-af59-56403ecb30fa\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") "
Oct 30 19:30:08.010858 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.010840    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/62577a04-5c57-4428-af59-56403ecb30fa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62577a04-5c57-4428-af59-56403ecb30fa-istiod-ca-cert\") pod \"62577a04-5c57-4428-af59-56403ecb30fa\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") "
Oct 30 19:30:08.010978 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.010963    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-credential-socket\") pod \"62577a04-5c57-4428-af59-56403ecb30fa\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") "
Oct 30 19:30:08.011227 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.011207    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62577a04-5c57-4428-af59-56403ecb30fa-kube-api-access-slgkl\" (UniqueName: \"kubernetes.io/projected/62577a04-5c57-4428-af59-56403ecb30fa-kube-api-access-slgkl\") pod \"62577a04-5c57-4428-af59-56403ecb30fa\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") "
Oct 30 19:30:08.011391 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.011374    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62577a04-5c57-4428-af59-56403ecb30fa-istio-token\" (UniqueName: \"kubernetes.io/projected/62577a04-5c57-4428-af59-56403ecb30fa-istio-token\") pod \"62577a04-5c57-4428-af59-56403ecb30fa\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") "
Oct 30 19:30:08.011553 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.011535    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-istio-data\") pod \"62577a04-5c57-4428-af59-56403ecb30fa\" (UID: \"62577a04-5c57-4428-af59-56403ecb30fa\") "
Oct 30 19:30:08.012674 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.012653    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:08.013141 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.013118    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-istio-data" pod "62577a04-5c57-4428-af59-56403ecb30fa" (UID: "62577a04-5c57-4428-af59-56403ecb30fa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:08.013529 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.013508    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-workload-certs" pod "62577a04-5c57-4428-af59-56403ecb30fa" (UID: "62577a04-5c57-4428-af59-56403ecb30fa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:08.021264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.019080    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62577a04-5c57-4428-af59-56403ecb30fa-istiod-ca-cert" pod "62577a04-5c57-4428-af59-56403ecb30fa" (UID: "62577a04-5c57-4428-af59-56403ecb30fa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:08.021372 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.021269    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-workload-socket" pod "62577a04-5c57-4428-af59-56403ecb30fa" (UID: "62577a04-5c57-4428-af59-56403ecb30fa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:08.021513 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.021488    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-credential-socket" pod "62577a04-5c57-4428-af59-56403ecb30fa" (UID: "62577a04-5c57-4428-af59-56403ecb30fa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:08.104190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.103351    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn"]
Oct 30 19:30:08.104536 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.104498    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/62577a04-5c57-4428-af59-56403ecb30fa-istio-podinfo" pod "62577a04-5c57-4428-af59-56403ecb30fa" (UID: "62577a04-5c57-4428-af59-56403ecb30fa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:30:08.104765 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.104744    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85"
Oct 30 19:30:08.124208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.121122    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:08.124208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.121154    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62577a04-5c57-4428-af59-56403ecb30fa-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:08.124208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.121187    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:08.124208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.121202    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62577a04-5c57-4428-af59-56403ecb30fa-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:08.124208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.121223    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:08.124208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.121236    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62577a04-5c57-4428-af59-56403ecb30fa-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:08.124208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.123636    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62577a04-5c57-4428-af59-56403ecb30fa-istio-token" pod "62577a04-5c57-4428-af59-56403ecb30fa" (UID: "62577a04-5c57-4428-af59-56403ecb30fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:08.124208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.123891    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62577a04-5c57-4428-af59-56403ecb30fa-kube-api-access-slgkl" pod "62577a04-5c57-4428-af59-56403ecb30fa" (UID: "62577a04-5c57-4428-af59-56403ecb30fa"). InnerVolumeSpecName "kube-api-access-slgkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:08.127284 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.125527    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw"]
Oct 30 19:30:08.127284 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.125848    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn"
Oct 30 19:30:08.139345 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.139323    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl"]
Oct 30 19:30:08.139537 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.139525    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw"
Oct 30 19:30:08.164916 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.164893    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf"]
Oct 30 19:30:08.166465 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.166238    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl"
Oct 30 19:30:08.195931 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.175310    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp"]
Oct 30 19:30:08.195931 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.175395    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf"
Oct 30 19:30:08.195931 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.188796    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc"]
Oct 30 19:30:08.195931 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.188892    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp"
Oct 30 19:30:08.195931 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.195683    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q"]
Oct 30 19:30:08.197193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.196985    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc"
Oct 30 19:30:08.201944 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.201917    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4"]
Oct 30 19:30:08.204902 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.204449    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q"
Oct 30 19:30:08.207694 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.207673    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85"]
Oct 30 19:30:08.207810 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.207713    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn"]
Oct 30 19:30:08.207810 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.207744    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw"]
Oct 30 19:30:08.207810 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.207748    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-ef5dca\""
Oct 30 19:30:08.207810 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.207768    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl"]
Oct 30 19:30:08.207810 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.207802    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc"]
Oct 30 19:30:08.208062 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.207822    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-ef5dca-dockercfg-74cps\""
Oct 30 19:30:08.208062 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.207826    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q"]
Oct 30 19:30:08.208062 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.207851    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4"
Oct 30 19:30:08.208580 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.208561    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp"]
Oct 30 19:30:08.213603 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.213587    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf"]
Oct 30 19:30:08.217740 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.217722    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4"]
Oct 30 19:30:08.222648 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.222618    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9c8d\" (UniqueName: \"kubernetes.io/projected/ce341e1c-1cbb-4ecc-8fda-c9088901b276-kube-api-access-q9c8d\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn\" (UID: \"ce341e1c-1cbb-4ecc-8fda-c9088901b276\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn"
Oct 30 19:30:08.222744 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.222670    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60537a63-2e4e-4441-a28e-2389fe0d0e93-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw\" (UID: \"60537a63-2e4e-4441-a28e-2389fe0d0e93\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw"
Oct 30 19:30:08.222814 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.222796    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f17debbb-3dd6-47a2-8799-376d5735185c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85\" (UID: \"f17debbb-3dd6-47a2-8799-376d5735185c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85"
Oct 30 19:30:08.222993 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.222977    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce341e1c-1cbb-4ecc-8fda-c9088901b276-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn\" (UID: \"ce341e1c-1cbb-4ecc-8fda-c9088901b276\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn"
Oct 30 19:30:08.223037 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.223019    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl5l8\" (UniqueName: \"kubernetes.io/projected/f17debbb-3dd6-47a2-8799-376d5735185c-kube-api-access-kl5l8\") pod \"maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85\" (UID: \"f17debbb-3dd6-47a2-8799-376d5735185c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85"
Oct 30 19:30:08.223157 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.223143    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggtxs\" (UniqueName: \"kubernetes.io/projected/60537a63-2e4e-4441-a28e-2389fe0d0e93-kube-api-access-ggtxs\") pod \"maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw\" (UID: \"60537a63-2e4e-4441-a28e-2389fe0d0e93\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw"
Oct 30 19:30:08.223365 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.223345    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slgkl\" (UniqueName: \"kubernetes.io/projected/62577a04-5c57-4428-af59-56403ecb30fa-kube-api-access-slgkl\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:08.223416 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.223370    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62577a04-5c57-4428-af59-56403ecb30fa-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:08.307928 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.307900    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:30:08.324998 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.324970    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wk2b\" (UniqueName: \"kubernetes.io/projected/a22cffda-4a00-4d56-a975-6ad1ab789847-kube-api-access-6wk2b\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc\" (UID: \"a22cffda-4a00-4d56-a975-6ad1ab789847\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc"
Oct 30 19:30:08.325132 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.325098    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a22cffda-4a00-4d56-a975-6ad1ab789847-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc\" (UID: \"a22cffda-4a00-4d56-a975-6ad1ab789847\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc"
Oct 30 19:30:08.325261 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.325148    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9c8d\" (UniqueName: \"kubernetes.io/projected/ce341e1c-1cbb-4ecc-8fda-c9088901b276-kube-api-access-q9c8d\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn\" (UID: \"ce341e1c-1cbb-4ecc-8fda-c9088901b276\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn"
Oct 30 19:30:08.325261 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.325233    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60537a63-2e4e-4441-a28e-2389fe0d0e93-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw\" (UID: \"60537a63-2e4e-4441-a28e-2389fe0d0e93\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw"
Oct 30 19:30:08.325442 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.325301    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5m2w\" (UniqueName: \"kubernetes.io/projected/23ed05e1-299a-4a05-b555-8a2966d04280-kube-api-access-f5m2w\") pod \"payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q\" (UID: \"23ed05e1-299a-4a05-b555-8a2966d04280\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q"
Oct 30 19:30:08.325442 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.325338    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f17debbb-3dd6-47a2-8799-376d5735185c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85\" (UID: \"f17debbb-3dd6-47a2-8799-376d5735185c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85"
Oct 30 19:30:08.325442 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.325380    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pv48\" (UniqueName: \"kubernetes.io/projected/27521cd5-2e0a-4836-af74-6a96ea61ba6d-kube-api-access-4pv48\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl\" (UID: \"27521cd5-2e0a-4836-af74-6a96ea61ba6d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl"
Oct 30 19:30:08.326570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.325554    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxcrr\" (UniqueName: \"kubernetes.io/projected/cc886570-4bd4-4f6d-8d49-2bcb11921b94-kube-api-access-cxcrr\") pod \"maas-api-revoke-sub-2596eef9-rkpz4\" (UID: \"cc886570-4bd4-4f6d-8d49-2bcb11921b94\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4"
Oct 30 19:30:08.326570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.325663    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/714dacfe-8ec6-4cbc-8aed-534f8ad30f2f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf\" (UID: \"714dacfe-8ec6-4cbc-8aed-534f8ad30f2f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf"
Oct 30 19:30:08.326570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.325731    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d5c4eb6-06df-4544-84de-78b3c8b0f4e9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp\" (UID: \"2d5c4eb6-06df-4544-84de-78b3c8b0f4e9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp"
Oct 30 19:30:08.326570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.325818    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce341e1c-1cbb-4ecc-8fda-c9088901b276-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn\" (UID: \"ce341e1c-1cbb-4ecc-8fda-c9088901b276\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn"
Oct 30 19:30:08.326570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.325856    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl5l8\" (UniqueName: \"kubernetes.io/projected/f17debbb-3dd6-47a2-8799-376d5735185c-kube-api-access-kl5l8\") pod \"maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85\" (UID: \"f17debbb-3dd6-47a2-8799-376d5735185c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85"
Oct 30 19:30:08.326570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.325885    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc886570-4bd4-4f6d-8d49-2bcb11921b94-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2596eef9-rkpz4\" (UID: \"cc886570-4bd4-4f6d-8d49-2bcb11921b94\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4"
Oct 30 19:30:08.326570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.325927    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc886570-4bd4-4f6d-8d49-2bcb11921b94-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2596eef9-rkpz4\" (UID: \"cc886570-4bd4-4f6d-8d49-2bcb11921b94\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4"
Oct 30 19:30:08.326570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.325999    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27521cd5-2e0a-4836-af74-6a96ea61ba6d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl\" (UID: \"27521cd5-2e0a-4836-af74-6a96ea61ba6d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl"
Oct 30 19:30:08.326570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.326076    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggtxs\" (UniqueName: \"kubernetes.io/projected/60537a63-2e4e-4441-a28e-2389fe0d0e93-kube-api-access-ggtxs\") pod \"maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw\" (UID: \"60537a63-2e4e-4441-a28e-2389fe0d0e93\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw"
Oct 30 19:30:08.326570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.326314    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mgf2\" (UniqueName: \"kubernetes.io/projected/714dacfe-8ec6-4cbc-8aed-534f8ad30f2f-kube-api-access-8mgf2\") pod \"maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf\" (UID: \"714dacfe-8ec6-4cbc-8aed-534f8ad30f2f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf"
Oct 30 19:30:08.326570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.326382    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23ed05e1-299a-4a05-b555-8a2966d04280-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q\" (UID: \"23ed05e1-299a-4a05-b555-8a2966d04280\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q"
Oct 30 19:30:08.326570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.326441    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsnvj\" (UniqueName: \"kubernetes.io/projected/2d5c4eb6-06df-4544-84de-78b3c8b0f4e9-kube-api-access-lsnvj\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp\" (UID: \"2d5c4eb6-06df-4544-84de-78b3c8b0f4e9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp"
Oct 30 19:30:08.327230 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.326579    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f17debbb-3dd6-47a2-8799-376d5735185c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85\" (UID: \"f17debbb-3dd6-47a2-8799-376d5735185c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85"
Oct 30 19:30:08.327230 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.326775    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60537a63-2e4e-4441-a28e-2389fe0d0e93-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw\" (UID: \"60537a63-2e4e-4441-a28e-2389fe0d0e93\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw"
Oct 30 19:30:08.327230 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.327164    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce341e1c-1cbb-4ecc-8fda-c9088901b276-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn\" (UID: \"ce341e1c-1cbb-4ecc-8fda-c9088901b276\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn"
Oct 30 19:30:08.337611 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.337588    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9c8d\" (UniqueName: \"kubernetes.io/projected/ce341e1c-1cbb-4ecc-8fda-c9088901b276-kube-api-access-q9c8d\") pod \"maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn\" (UID: \"ce341e1c-1cbb-4ecc-8fda-c9088901b276\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn"
Oct 30 19:30:08.337721 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.337702    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggtxs\" (UniqueName: \"kubernetes.io/projected/60537a63-2e4e-4441-a28e-2389fe0d0e93-kube-api-access-ggtxs\") pod \"maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw\" (UID: \"60537a63-2e4e-4441-a28e-2389fe0d0e93\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw"
Oct 30 19:30:08.341339 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.341306    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl5l8\" (UniqueName: \"kubernetes.io/projected/f17debbb-3dd6-47a2-8799-376d5735185c-kube-api-access-kl5l8\") pod \"maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85\" (UID: \"f17debbb-3dd6-47a2-8799-376d5735185c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85"
Oct 30 19:30:08.431198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.427850    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pv48\" (UniqueName: \"kubernetes.io/projected/27521cd5-2e0a-4836-af74-6a96ea61ba6d-kube-api-access-4pv48\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl\" (UID: \"27521cd5-2e0a-4836-af74-6a96ea61ba6d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl"
Oct 30 19:30:08.431198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.427898    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxcrr\" (UniqueName: \"kubernetes.io/projected/cc886570-4bd4-4f6d-8d49-2bcb11921b94-kube-api-access-cxcrr\") pod \"maas-api-revoke-sub-2596eef9-rkpz4\" (UID: \"cc886570-4bd4-4f6d-8d49-2bcb11921b94\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4"
Oct 30 19:30:08.431198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.427989    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/714dacfe-8ec6-4cbc-8aed-534f8ad30f2f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf\" (UID: \"714dacfe-8ec6-4cbc-8aed-534f8ad30f2f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf"
Oct 30 19:30:08.431198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.428041    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d5c4eb6-06df-4544-84de-78b3c8b0f4e9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp\" (UID: \"2d5c4eb6-06df-4544-84de-78b3c8b0f4e9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp"
Oct 30 19:30:08.431198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.428110    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc886570-4bd4-4f6d-8d49-2bcb11921b94-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2596eef9-rkpz4\" (UID: \"cc886570-4bd4-4f6d-8d49-2bcb11921b94\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4"
Oct 30 19:30:08.431198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.428148    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc886570-4bd4-4f6d-8d49-2bcb11921b94-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2596eef9-rkpz4\" (UID: \"cc886570-4bd4-4f6d-8d49-2bcb11921b94\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4"
Oct 30 19:30:08.431198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.428235    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27521cd5-2e0a-4836-af74-6a96ea61ba6d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl\" (UID: \"27521cd5-2e0a-4836-af74-6a96ea61ba6d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl"
Oct 30 19:30:08.431198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.428299    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mgf2\" (UniqueName: \"kubernetes.io/projected/714dacfe-8ec6-4cbc-8aed-534f8ad30f2f-kube-api-access-8mgf2\") pod \"maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf\" (UID: \"714dacfe-8ec6-4cbc-8aed-534f8ad30f2f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf"
Oct 30 19:30:08.431198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.428338    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23ed05e1-299a-4a05-b555-8a2966d04280-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q\" (UID: \"23ed05e1-299a-4a05-b555-8a2966d04280\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q"
Oct 30 19:30:08.431198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.428377    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsnvj\" (UniqueName: \"kubernetes.io/projected/2d5c4eb6-06df-4544-84de-78b3c8b0f4e9-kube-api-access-lsnvj\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp\" (UID: \"2d5c4eb6-06df-4544-84de-78b3c8b0f4e9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp"
Oct 30 19:30:08.431198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.428425    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wk2b\" (UniqueName: \"kubernetes.io/projected/a22cffda-4a00-4d56-a975-6ad1ab789847-kube-api-access-6wk2b\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc\" (UID: \"a22cffda-4a00-4d56-a975-6ad1ab789847\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc"
Oct 30 19:30:08.431198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.428501    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a22cffda-4a00-4d56-a975-6ad1ab789847-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc\" (UID: \"a22cffda-4a00-4d56-a975-6ad1ab789847\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc"
Oct 30 19:30:08.431198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.428611    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5m2w\" (UniqueName: \"kubernetes.io/projected/23ed05e1-299a-4a05-b555-8a2966d04280-kube-api-access-f5m2w\") pod \"payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q\" (UID: \"23ed05e1-299a-4a05-b555-8a2966d04280\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q"
Oct 30 19:30:08.431198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.429946    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/714dacfe-8ec6-4cbc-8aed-534f8ad30f2f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf\" (UID: \"714dacfe-8ec6-4cbc-8aed-534f8ad30f2f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf"
Oct 30 19:30:08.431198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.430564    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d5c4eb6-06df-4544-84de-78b3c8b0f4e9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp\" (UID: \"2d5c4eb6-06df-4544-84de-78b3c8b0f4e9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp"
Oct 30 19:30:08.484753 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.474153    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27521cd5-2e0a-4836-af74-6a96ea61ba6d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl\" (UID: \"27521cd5-2e0a-4836-af74-6a96ea61ba6d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl"
Oct 30 19:30:08.484753 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.483028    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a22cffda-4a00-4d56-a975-6ad1ab789847-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc\" (UID: \"a22cffda-4a00-4d56-a975-6ad1ab789847\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc"
Oct 30 19:30:08.499285 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.499256    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-ef5dca\""
Oct 30 19:30:08.499534 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.499520    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:30:08.507265 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.506892    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23ed05e1-299a-4a05-b555-8a2966d04280-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q\" (UID: \"23ed05e1-299a-4a05-b555-8a2966d04280\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q"
Oct 30 19:30:08.507995 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.507516    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc886570-4bd4-4f6d-8d49-2bcb11921b94-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-2596eef9-rkpz4\" (UID: \"cc886570-4bd4-4f6d-8d49-2bcb11921b94\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4"
Oct 30 19:30:08.508496 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.508248    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxcrr\" (UniqueName: \"kubernetes.io/projected/cc886570-4bd4-4f6d-8d49-2bcb11921b94-kube-api-access-cxcrr\") pod \"maas-api-revoke-sub-2596eef9-rkpz4\" (UID: \"cc886570-4bd4-4f6d-8d49-2bcb11921b94\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4"
Oct 30 19:30:08.508496 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.508312    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn"
Oct 30 19:30:08.509410 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.509020    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5m2w\" (UniqueName: \"kubernetes.io/projected/23ed05e1-299a-4a05-b555-8a2966d04280-kube-api-access-f5m2w\") pod \"payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q\" (UID: \"23ed05e1-299a-4a05-b555-8a2966d04280\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q"
Oct 30 19:30:08.513038 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.512966    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85"
Oct 30 19:30:08.517332 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.517306    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsnvj\" (UniqueName: \"kubernetes.io/projected/2d5c4eb6-06df-4544-84de-78b3c8b0f4e9-kube-api-access-lsnvj\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp\" (UID: \"2d5c4eb6-06df-4544-84de-78b3c8b0f4e9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp"
Oct 30 19:30:08.536833 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.535017    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc886570-4bd4-4f6d-8d49-2bcb11921b94-maas-api-xks-ca\") pod \"maas-api-revoke-sub-2596eef9-rkpz4\" (UID: \"cc886570-4bd4-4f6d-8d49-2bcb11921b94\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4"
Oct 30 19:30:08.576320 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.575939    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wk2b\" (UniqueName: \"kubernetes.io/projected/a22cffda-4a00-4d56-a975-6ad1ab789847-kube-api-access-6wk2b\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc\" (UID: \"a22cffda-4a00-4d56-a975-6ad1ab789847\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc"
Oct 30 19:30:08.601299 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.585401    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw"
Oct 30 19:30:08.601299 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.598651    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pv48\" (UniqueName: \"kubernetes.io/projected/27521cd5-2e0a-4836-af74-6a96ea61ba6d-kube-api-access-4pv48\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl\" (UID: \"27521cd5-2e0a-4836-af74-6a96ea61ba6d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl"
Oct 30 19:30:08.601299 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.600085    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mgf2\" (UniqueName: \"kubernetes.io/projected/714dacfe-8ec6-4cbc-8aed-534f8ad30f2f-kube-api-access-8mgf2\") pod \"maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf\" (UID: \"714dacfe-8ec6-4cbc-8aed-534f8ad30f2f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf"
Oct 30 19:30:08.627973 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.627909    4524 generic.go:356] "Generic (PLEG): container finished" podID="652b5102-d21a-4673-b955-fb005179323e" containerID="dfbcd68013e45941e8e5e5b5ba2fc5f02509743d543fd7d3ceca51e097a0ef36" exitCode=60
Oct 30 19:30:08.628114 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.628006    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9" event={"ID":"652b5102-d21a-4673-b955-fb005179323e","Type":"ContainerDied","Data":"dfbcd68013e45941e8e5e5b5ba2fc5f02509743d543fd7d3ceca51e097a0ef36"}
Oct 30 19:30:08.628437 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.628343    4524 scope.go:122] "RemoveContainer" containerID="dfbcd68013e45941e8e5e5b5ba2fc5f02509743d543fd7d3ceca51e097a0ef36"
Oct 30 19:30:08.630239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.630155    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-ef5dca-dockercfg-74cps\""
Oct 30 19:30:08.631703 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.631677    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4"
Oct 30 19:30:08.633052 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.632292    4524 generic.go:356] "Generic (PLEG): container finished" podID="7543e764-a757-4962-8342-f6d9d9e0f368" containerID="24d9ae6fa79852055034ca9156a599a5a46e6caa466ea0cc570ab8385b459fc4" exitCode=60
Oct 30 19:30:08.633052 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.632607    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp" event={"ID":"7543e764-a757-4962-8342-f6d9d9e0f368","Type":"ContainerDied","Data":"24d9ae6fa79852055034ca9156a599a5a46e6caa466ea0cc570ab8385b459fc4"}
Oct 30 19:30:08.633052 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.632695    4524 scope.go:122] "RemoveContainer" containerID="24d9ae6fa79852055034ca9156a599a5a46e6caa466ea0cc570ab8385b459fc4"
Oct 30 19:30:08.635601 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.635575    4524 generic.go:356] "Generic (PLEG): container finished" podID="62577a04-5c57-4428-af59-56403ecb30fa" containerID="22cb7997b22b713038f32a165d352526594fe90953cb8141a694e197b4892fb4" exitCode=0
Oct 30 19:30:08.635729 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.635676    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24" event={"ID":"62577a04-5c57-4428-af59-56403ecb30fa","Type":"ContainerDied","Data":"22cb7997b22b713038f32a165d352526594fe90953cb8141a694e197b4892fb4"}
Oct 30 19:30:08.635888 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.635871    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"
Oct 30 19:30:08.635950 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.635889    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24" event={"ID":"62577a04-5c57-4428-af59-56403ecb30fa","Type":"ContainerDied","Data":"8478547abefacfba7e9d5ca7a11f49340a2f3c9ae707c56140be523ff683acc4"}
Oct 30 19:30:08.635950 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.635925    4524 scope.go:122] "RemoveContainer" containerID="22cb7997b22b713038f32a165d352526594fe90953cb8141a694e197b4892fb4"
Oct 30 19:30:08.638325 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.638306    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q"
Oct 30 19:30:08.643227 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.643001    4524 generic.go:356] "Generic (PLEG): container finished" podID="3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d" containerID="0afe64e4d35728a949f07f0aa2c166e5da7e0589c23396e8de3cdf756225026d" exitCode=60
Oct 30 19:30:08.643227 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.643083    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x" event={"ID":"3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d","Type":"ContainerDied","Data":"0afe64e4d35728a949f07f0aa2c166e5da7e0589c23396e8de3cdf756225026d"}
Oct 30 19:30:08.643366 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.643345    4524 scope.go:122] "RemoveContainer" containerID="0afe64e4d35728a949f07f0aa2c166e5da7e0589c23396e8de3cdf756225026d"
Oct 30 19:30:08.646281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.646245    4524 generic.go:356] "Generic (PLEG): container finished" podID="c436fb87-4351-4a27-9286-216883df85cd" containerID="a1ae496a2c7cc537460f8d23f79a6691ba08d5d4703a2ba33cae67db9c2cf185" exitCode=60
Oct 30 19:30:08.646391 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.646316    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt" event={"ID":"c436fb87-4351-4a27-9286-216883df85cd","Type":"ContainerDied","Data":"a1ae496a2c7cc537460f8d23f79a6691ba08d5d4703a2ba33cae67db9c2cf185"}
Oct 30 19:30:08.646501 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.646485    4524 scope.go:122] "RemoveContainer" containerID="a1ae496a2c7cc537460f8d23f79a6691ba08d5d4703a2ba33cae67db9c2cf185"
Oct 30 19:30:08.663646 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.663545    4524 scope.go:122] "RemoveContainer" containerID="22cb7997b22b713038f32a165d352526594fe90953cb8141a694e197b4892fb4"
Oct 30 19:30:08.679192 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:08.677296    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22cb7997b22b713038f32a165d352526594fe90953cb8141a694e197b4892fb4\": container with ID starting with 22cb7997b22b713038f32a165d352526594fe90953cb8141a694e197b4892fb4 not found: ID does not exist" containerID="22cb7997b22b713038f32a165d352526594fe90953cb8141a694e197b4892fb4"
Oct 30 19:30:08.679192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.677336    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22cb7997b22b713038f32a165d352526594fe90953cb8141a694e197b4892fb4"} err="failed to get container status \"22cb7997b22b713038f32a165d352526594fe90953cb8141a694e197b4892fb4\": rpc error: code = NotFound desc = could not find container \"22cb7997b22b713038f32a165d352526594fe90953cb8141a694e197b4892fb4\": container with ID starting with 22cb7997b22b713038f32a165d352526594fe90953cb8141a694e197b4892fb4 not found: ID does not exist"
Oct 30 19:30:08.689578 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.688376    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl"
Oct 30 19:30:08.721281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.721044    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp"
Oct 30 19:30:08.723222 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.721902    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf"
Oct 30 19:30:08.762441 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:08.762296    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc"
Oct 30 19:30:09.008992 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.008948    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"]
Oct 30 19:30:09.016849 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.016824    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-48b1a6a7-openshift-default-78f6598f5c-2mf24"]
Oct 30 19:30:09.137592 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.131248    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85"]
Oct 30 19:30:09.400764 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.400736    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw"]
Oct 30 19:30:09.406157 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.406132    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn"]
Oct 30 19:30:09.459666 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.459630    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4"]
Oct 30 19:30:09.609111 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.607333    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl"]
Oct 30 19:30:09.679883 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.673931    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q"]
Oct 30 19:30:09.685436 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.684458    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc"]
Oct 30 19:30:09.698353 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.698331    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf"]
Oct 30 19:30:09.705569 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.705543    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp"]
Oct 30 19:30:09.712243 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:30:09.712215    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d5c4eb6_06df_4544_84de_78b3c8b0f4e9.slice/crio-36edfbb142e91d3533b4d590f2b31c7b9fe33cd85fc539e894cde3e20a473edc WatchSource:0}: Error finding container 36edfbb142e91d3533b4d590f2b31c7b9fe33cd85fc539e894cde3e20a473edc: Status 404 returned error can't find the container with id 36edfbb142e91d3533b4d590f2b31c7b9fe33cd85fc539e894cde3e20a473edc
Oct 30 19:30:09.719404 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.719366    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4" event={"ID":"cc886570-4bd4-4f6d-8d49-2bcb11921b94","Type":"ContainerStarted","Data":"f37e850becd9eebbf2ed67607eee46a2166a6157f8d7aae37890427c0a089921"}
Oct 30 19:30:09.736227 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.736196    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn" event={"ID":"ce341e1c-1cbb-4ecc-8fda-c9088901b276","Type":"ContainerStarted","Data":"264141812a36b0e8af778d0c9ecb1da9eb8ceafbab1eb878d426be846c3156a4"}
Oct 30 19:30:09.740919 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.740261    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9" event={"ID":"652b5102-d21a-4673-b955-fb005179323e","Type":"ContainerStarted","Data":"76fd6a0f016f9aa7763f1bec2b8168659a42656eede582ee7d27d4462fd662e5"}
Oct 30 19:30:09.753891 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.753867    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl" event={"ID":"27521cd5-2e0a-4836-af74-6a96ea61ba6d","Type":"ContainerStarted","Data":"56eea5c104f050aed74c3803f25c6761b14731d0d0d39e04b45abee12a9a5402"}
Oct 30 19:30:09.758970 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.758942    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw" event={"ID":"60537a63-2e4e-4441-a28e-2389fe0d0e93","Type":"ContainerStarted","Data":"d9c4c08f86206c59c55e667768a8ac1708b48561ac57a5f67d0e2e1d93f4a756"}
Oct 30 19:30:09.762289 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.762238    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt" event={"ID":"c436fb87-4351-4a27-9286-216883df85cd","Type":"ContainerStarted","Data":"5426cab574c22fa1020164e69da8935f2819b0e710c84ae0ab0e7970cab71506"}
Oct 30 19:30:09.764101 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.764050    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9" podStartSLOduration=4.28979184 podStartE2EDuration="9.764032695s" totalImagesPullingTime="5.474240855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:09.758824916 +0000 UTC m=+1441.499815996" watchObservedRunningTime="2026-10-30 19:30:09.764032695 +0000 UTC m=+1441.505023776"
Oct 30 19:30:09.764642 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.764591    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85" event={"ID":"f17debbb-3dd6-47a2-8799-376d5735185c","Type":"ContainerStarted","Data":"e754acf620b409ab692ced92bfecd717e58aea8863dc350152ebf8527bb78be2"}
Oct 30 19:30:09.781299 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.780946    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x" podStartSLOduration=4.018526037 podStartE2EDuration="9.780927634s" totalImagesPullingTime="5.762401597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:09.776742456 +0000 UTC m=+1441.517733536" watchObservedRunningTime="2026-10-30 19:30:09.780927634 +0000 UTC m=+1441.521918714"
Oct 30 19:30:09.807468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:09.805608    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt" podStartSLOduration=4.187086643 podStartE2EDuration="9.805594225s" totalImagesPullingTime="5.618507582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:09.801759952 +0000 UTC m=+1441.542751031" watchObservedRunningTime="2026-10-30 19:30:09.805594225 +0000 UTC m=+1441.546585303"
Oct 30 19:30:10.483311 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.483277    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="62577a04-5c57-4428-af59-56403ecb30fa" path="/var/lib/kubelet/pods/62577a04-5c57-4428-af59-56403ecb30fa/volumes"
Oct 30 19:30:10.781005 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.780972    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4" event={"ID":"cc886570-4bd4-4f6d-8d49-2bcb11921b94","Type":"ContainerStarted","Data":"e9e21d4e476d3aca7bbdb5e19995e7ea39984cc1bb701e3e3c09806da21dcdd4"}
Oct 30 19:30:10.784293 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.784221    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn" event={"ID":"ce341e1c-1cbb-4ecc-8fda-c9088901b276","Type":"ContainerStarted","Data":"38adb2dee843852c3ec1566780493d53d5c8183a1d24b3d788d78299e689e0ff"}
Oct 30 19:30:10.786982 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.786951    4524 generic.go:356] "Generic (PLEG): container finished" podID="652b5102-d21a-4673-b955-fb005179323e" containerID="76fd6a0f016f9aa7763f1bec2b8168659a42656eede582ee7d27d4462fd662e5" exitCode=60
Oct 30 19:30:10.787094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.787040    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9" event={"ID":"652b5102-d21a-4673-b955-fb005179323e","Type":"ContainerDied","Data":"76fd6a0f016f9aa7763f1bec2b8168659a42656eede582ee7d27d4462fd662e5"}
Oct 30 19:30:10.787145 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.787127    4524 scope.go:122] "RemoveContainer" containerID="dfbcd68013e45941e8e5e5b5ba2fc5f02509743d543fd7d3ceca51e097a0ef36"
Oct 30 19:30:10.787347 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.787304    4524 scope.go:122] "RemoveContainer" containerID="76fd6a0f016f9aa7763f1bec2b8168659a42656eede582ee7d27d4462fd662e5"
Oct 30 19:30:10.787647 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:10.787627    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9_odh-ai-gateway-infra(652b5102-d21a-4673-b955-fb005179323e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9" podUID="652b5102-d21a-4673-b955-fb005179323e"
Oct 30 19:30:10.790074 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.790037    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q" event={"ID":"23ed05e1-299a-4a05-b555-8a2966d04280","Type":"ContainerStarted","Data":"bd706f3c0881ba7f65b83c0b75ee477533aab86444caf1c6a42d846ac1dca34b"}
Oct 30 19:30:10.790232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.790087    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q" event={"ID":"23ed05e1-299a-4a05-b555-8a2966d04280","Type":"ContainerStarted","Data":"9d7478c0783653c4a68195be6b63eccbe5abacc8296efd89e0dfdb6801a45211"}
Oct 30 19:30:10.790232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.790119    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q"
Oct 30 19:30:10.792402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.792376    4524 generic.go:356] "Generic (PLEG): container finished" podID="7543e764-a757-4962-8342-f6d9d9e0f368" containerID="c0ab990afb10ae4463e957a9700ca91164e8f6a1ae6f81bb4fa0f5557159e3a3" exitCode=60
Oct 30 19:30:10.792709 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.792691    4524 scope.go:122] "RemoveContainer" containerID="c0ab990afb10ae4463e957a9700ca91164e8f6a1ae6f81bb4fa0f5557159e3a3"
Oct 30 19:30:10.793056 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.792912    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp" event={"ID":"7543e764-a757-4962-8342-f6d9d9e0f368","Type":"ContainerDied","Data":"c0ab990afb10ae4463e957a9700ca91164e8f6a1ae6f81bb4fa0f5557159e3a3"}
Oct 30 19:30:10.793056 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:10.793009    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp_odh-ai-gateway-infra(7543e764-a757-4962-8342-f6d9d9e0f368)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp" podUID="7543e764-a757-4962-8342-f6d9d9e0f368"
Oct 30 19:30:10.794566 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.794542    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl" event={"ID":"27521cd5-2e0a-4836-af74-6a96ea61ba6d","Type":"ContainerStarted","Data":"7890fce46c36779fa0928563db875aa4d7eef9aad1a51b32d47a83bd9c57d544"}
Oct 30 19:30:10.796936 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.796917    4524 generic.go:356] "Generic (PLEG): container finished" podID="3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d" containerID="7dbfc3f919fdd1f49433d8069ce99555d7ba4b72902ce9e191ad651d0824f682" exitCode=60
Oct 30 19:30:10.797097 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.796993    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x" event={"ID":"3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d","Type":"ContainerDied","Data":"7dbfc3f919fdd1f49433d8069ce99555d7ba4b72902ce9e191ad651d0824f682"}
Oct 30 19:30:10.797354 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.797338    4524 scope.go:122] "RemoveContainer" containerID="7dbfc3f919fdd1f49433d8069ce99555d7ba4b72902ce9e191ad651d0824f682"
Oct 30 19:30:10.797706 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:10.797674    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x_odh-ai-gateway-infra(3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x" podUID="3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d"
Oct 30 19:30:10.798872 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.798823    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw" event={"ID":"60537a63-2e4e-4441-a28e-2389fe0d0e93","Type":"ContainerStarted","Data":"b175cd4be4e22c4c2bfc9e12e1c87008610fa28a6fa16d46a6a48940db1bedf1"}
Oct 30 19:30:10.801701 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.801681    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf" event={"ID":"714dacfe-8ec6-4cbc-8aed-534f8ad30f2f","Type":"ContainerStarted","Data":"1e96f43298e7f98fe1f771b738bacf87f88542e1bdad46f92c9866ba89e22128"}
Oct 30 19:30:10.801867 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.801841    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf" event={"ID":"714dacfe-8ec6-4cbc-8aed-534f8ad30f2f","Type":"ContainerStarted","Data":"c3d0cf4234344d4d008b04851adcdca8377e7847481ce8a24f37681d2994177c"}
Oct 30 19:30:10.804075 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.804031    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4" podStartSLOduration=7.80401662 podStartE2EDuration="7.80401662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:10.799202658 +0000 UTC m=+1442.540193745" watchObservedRunningTime="2026-10-30 19:30:10.80401662 +0000 UTC m=+1442.545007698"
Oct 30 19:30:10.805019 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.804997    4524 generic.go:356] "Generic (PLEG): container finished" podID="c436fb87-4351-4a27-9286-216883df85cd" containerID="5426cab574c22fa1020164e69da8935f2819b0e710c84ae0ab0e7970cab71506" exitCode=60
Oct 30 19:30:10.805107 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.805093    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt" event={"ID":"c436fb87-4351-4a27-9286-216883df85cd","Type":"ContainerDied","Data":"5426cab574c22fa1020164e69da8935f2819b0e710c84ae0ab0e7970cab71506"}
Oct 30 19:30:10.805446 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.805429    4524 scope.go:122] "RemoveContainer" containerID="5426cab574c22fa1020164e69da8935f2819b0e710c84ae0ab0e7970cab71506"
Oct 30 19:30:10.805747 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:10.805726    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt_odh-ai-gateway-infra(c436fb87-4351-4a27-9286-216883df85cd)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt" podUID="c436fb87-4351-4a27-9286-216883df85cd"
Oct 30 19:30:10.805842 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.805829    4524 scope.go:122] "RemoveContainer" containerID="24d9ae6fa79852055034ca9156a599a5a46e6caa466ea0cc570ab8385b459fc4"
Oct 30 19:30:10.813349 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.808849    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp" event={"ID":"2d5c4eb6-06df-4544-84de-78b3c8b0f4e9","Type":"ContainerStarted","Data":"d7c15f5d043272b394114677117a0609d47b25f3d8c8cf1ea2fb577477fe6ac4"}
Oct 30 19:30:10.813349 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.809002    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp" event={"ID":"2d5c4eb6-06df-4544-84de-78b3c8b0f4e9","Type":"ContainerStarted","Data":"36edfbb142e91d3533b4d590f2b31c7b9fe33cd85fc539e894cde3e20a473edc"}
Oct 30 19:30:10.813349 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.811500    4524 generic.go:356] "Generic (PLEG): container finished" podID="f17debbb-3dd6-47a2-8799-376d5735185c" containerID="1604c39c1469d7a0497bd7d21fccdd118c8ce10ba39e751df8dab248576bd7c2" exitCode=60
Oct 30 19:30:10.813349 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.811591    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85" event={"ID":"f17debbb-3dd6-47a2-8799-376d5735185c","Type":"ContainerDied","Data":"1604c39c1469d7a0497bd7d21fccdd118c8ce10ba39e751df8dab248576bd7c2"}
Oct 30 19:30:10.813349 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.811945    4524 scope.go:122] "RemoveContainer" containerID="1604c39c1469d7a0497bd7d21fccdd118c8ce10ba39e751df8dab248576bd7c2"
Oct 30 19:30:10.817791 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.817765    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc" event={"ID":"a22cffda-4a00-4d56-a975-6ad1ab789847","Type":"ContainerStarted","Data":"bfc10ecdd231db5aaaca9c9160ee8e8a43316ca24edba23ac4ab9e10d40ca4c6"}
Oct 30 19:30:10.818251 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.818042    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc" event={"ID":"a22cffda-4a00-4d56-a975-6ad1ab789847","Type":"ContainerStarted","Data":"bb9d91a7e0d08544c5362b003813740e717898d501b7255c6aa480ae6b341227"}
Oct 30 19:30:10.836302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.836282    4524 scope.go:122] "RemoveContainer" containerID="0afe64e4d35728a949f07f0aa2c166e5da7e0589c23396e8de3cdf756225026d"
Oct 30 19:30:10.840444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.840397    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q" podStartSLOduration=9.840384669 podStartE2EDuration="9.840384669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:10.8364332 +0000 UTC m=+1442.577424280" watchObservedRunningTime="2026-10-30 19:30:10.840384669 +0000 UTC m=+1442.581375751"
Oct 30 19:30:10.853714 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.853693    4524 scope.go:122] "RemoveContainer" containerID="a1ae496a2c7cc537460f8d23f79a6691ba08d5d4703a2ba33cae67db9c2cf185"
Oct 30 19:30:10.861364 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.861317    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw" podStartSLOduration=10.861304252 podStartE2EDuration="10.861304252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:10.85529902 +0000 UTC m=+1442.596290111" watchObservedRunningTime="2026-10-30 19:30:10.861304252 +0000 UTC m=+1442.602295331"
Oct 30 19:30:10.906548 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.906432    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl" podStartSLOduration=10.906417315 podStartE2EDuration="10.906417315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:10.902802294 +0000 UTC m=+1442.643793374" watchObservedRunningTime="2026-10-30 19:30:10.906417315 +0000 UTC m=+1442.647408425"
Oct 30 19:30:10.927219 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:10.927157    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn" podStartSLOduration=10.92714385 podStartE2EDuration="10.92714385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:10.921043551 +0000 UTC m=+1442.662034663" watchObservedRunningTime="2026-10-30 19:30:10.92714385 +0000 UTC m=+1442.668134928"
Oct 30 19:30:11.218391 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.216990    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp" podStartSLOduration=11.216963308 podStartE2EDuration="11.216963308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:11.086301253 +0000 UTC m=+1442.827292332" watchObservedRunningTime="2026-10-30 19:30:11.216963308 +0000 UTC m=+1442.957954387"
Oct 30 19:30:11.224697 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.224641    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc" podStartSLOduration=11.224619634 podStartE2EDuration="11.224619634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:11.110278258 +0000 UTC m=+1442.851269340" watchObservedRunningTime="2026-10-30 19:30:11.224619634 +0000 UTC m=+1442.965610715"
Oct 30 19:30:11.236466 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.236416    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf" podStartSLOduration=11.236400089 podStartE2EDuration="11.236400089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:11.130585474 +0000 UTC m=+1442.871576553" watchObservedRunningTime="2026-10-30 19:30:11.236400089 +0000 UTC m=+1442.977391159"
Oct 30 19:30:11.566766 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.566744    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:30:11.628798 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.628747    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-workload-certs\") pod \"83f517e5-9b8e-425a-845b-56253704cd78\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") "
Oct 30 19:30:11.628954 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.628806    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-istio-envoy\") pod \"83f517e5-9b8e-425a-845b-56253704cd78\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") "
Oct 30 19:30:11.628954 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.628848    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-workload-socket\") pod \"83f517e5-9b8e-425a-845b-56253704cd78\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") "
Oct 30 19:30:11.629078 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.629043    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/83f517e5-9b8e-425a-845b-56253704cd78-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83f517e5-9b8e-425a-845b-56253704cd78-istio-podinfo\") pod \"83f517e5-9b8e-425a-845b-56253704cd78\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") "
Oct 30 19:30:11.629139 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.629124    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83f517e5-9b8e-425a-845b-56253704cd78-kube-api-access-gtdbz\" (UniqueName: \"kubernetes.io/projected/83f517e5-9b8e-425a-845b-56253704cd78-kube-api-access-gtdbz\") pod \"83f517e5-9b8e-425a-845b-56253704cd78\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") "
Oct 30 19:30:11.629340 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.629317    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/83f517e5-9b8e-425a-845b-56253704cd78-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83f517e5-9b8e-425a-845b-56253704cd78-istiod-ca-cert\") pod \"83f517e5-9b8e-425a-845b-56253704cd78\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") "
Oct 30 19:30:11.629420 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.629355    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-credential-socket\") pod \"83f517e5-9b8e-425a-845b-56253704cd78\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") "
Oct 30 19:30:11.629518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.629500    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-istio-data\") pod \"83f517e5-9b8e-425a-845b-56253704cd78\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") "
Oct 30 19:30:11.629584 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.629547    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/83f517e5-9b8e-425a-845b-56253704cd78-istio-token\" (UniqueName: \"kubernetes.io/projected/83f517e5-9b8e-425a-845b-56253704cd78-istio-token\") pod \"83f517e5-9b8e-425a-845b-56253704cd78\" (UID: \"83f517e5-9b8e-425a-845b-56253704cd78\") "
Oct 30 19:30:11.630021 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.629038    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-workload-certs" pod "83f517e5-9b8e-425a-845b-56253704cd78" (UID: "83f517e5-9b8e-425a-845b-56253704cd78"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:11.630021 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.629320    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-workload-socket" pod "83f517e5-9b8e-425a-845b-56253704cd78" (UID: "83f517e5-9b8e-425a-845b-56253704cd78"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:11.630285 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.630231    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-istio-data" pod "83f517e5-9b8e-425a-845b-56253704cd78" (UID: "83f517e5-9b8e-425a-845b-56253704cd78"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:11.630285 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.630246    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:11.630404 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.630293    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:11.630467 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.630449    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83f517e5-9b8e-425a-845b-56253704cd78-istiod-ca-cert" pod "83f517e5-9b8e-425a-845b-56253704cd78" (UID: "83f517e5-9b8e-425a-845b-56253704cd78"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:11.630518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.630503    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-credential-socket" pod "83f517e5-9b8e-425a-845b-56253704cd78" (UID: "83f517e5-9b8e-425a-845b-56253704cd78"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:11.632802 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.632774    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83f517e5-9b8e-425a-845b-56253704cd78-istio-token" pod "83f517e5-9b8e-425a-845b-56253704cd78" (UID: "83f517e5-9b8e-425a-845b-56253704cd78"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:11.635597 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.635557    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/83f517e5-9b8e-425a-845b-56253704cd78-istio-podinfo" pod "83f517e5-9b8e-425a-845b-56253704cd78" (UID: "83f517e5-9b8e-425a-845b-56253704cd78"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:30:11.638645 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.638591    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83f517e5-9b8e-425a-845b-56253704cd78-kube-api-access-gtdbz" pod "83f517e5-9b8e-425a-845b-56253704cd78" (UID: "83f517e5-9b8e-425a-845b-56253704cd78"). InnerVolumeSpecName "kube-api-access-gtdbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:11.639905 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.639869    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-istio-envoy" pod "83f517e5-9b8e-425a-845b-56253704cd78" (UID: "83f517e5-9b8e-425a-845b-56253704cd78"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:11.731673 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.731565    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:11.731673 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.731590    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/83f517e5-9b8e-425a-845b-56253704cd78-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:11.731673 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.731599    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:11.731673 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.731608    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/83f517e5-9b8e-425a-845b-56253704cd78-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:11.731673 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.731617    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtdbz\" (UniqueName: \"kubernetes.io/projected/83f517e5-9b8e-425a-845b-56253704cd78-kube-api-access-gtdbz\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:11.731673 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.731639    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/83f517e5-9b8e-425a-845b-56253704cd78-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:11.731673 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.731648    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/83f517e5-9b8e-425a-845b-56253704cd78-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:11.833369 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.833333    4524 generic.go:356] "Generic (PLEG): container finished" podID="83f517e5-9b8e-425a-845b-56253704cd78" containerID="b71879018815d86e282d8f0e69001291a1091babd9d025219863a4f39a8df76d" exitCode=0
Oct 30 19:30:11.833526 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.833393    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"
Oct 30 19:30:11.833526 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.833418    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9" event={"ID":"83f517e5-9b8e-425a-845b-56253704cd78","Type":"ContainerDied","Data":"b71879018815d86e282d8f0e69001291a1091babd9d025219863a4f39a8df76d"}
Oct 30 19:30:11.833526 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.833484    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9" event={"ID":"83f517e5-9b8e-425a-845b-56253704cd78","Type":"ContainerDied","Data":"e444c31ab6a997cc550eeb90aba60f33205fbb8928e41438f9d6023e1d49ebf2"}
Oct 30 19:30:11.833526 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.833514    4524 scope.go:122] "RemoveContainer" containerID="b71879018815d86e282d8f0e69001291a1091babd9d025219863a4f39a8df76d"
Oct 30 19:30:11.838798 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.838766    4524 generic.go:356] "Generic (PLEG): container finished" podID="60537a63-2e4e-4441-a28e-2389fe0d0e93" containerID="b175cd4be4e22c4c2bfc9e12e1c87008610fa28a6fa16d46a6a48940db1bedf1" exitCode=7
Oct 30 19:30:11.838927 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.838845    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw" event={"ID":"60537a63-2e4e-4441-a28e-2389fe0d0e93","Type":"ContainerDied","Data":"b175cd4be4e22c4c2bfc9e12e1c87008610fa28a6fa16d46a6a48940db1bedf1"}
Oct 30 19:30:11.839224 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.839121    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:30:11.839224 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.839125    4524 scope.go:122] "RemoveContainer" containerID="b175cd4be4e22c4c2bfc9e12e1c87008610fa28a6fa16d46a6a48940db1bedf1"
Oct 30 19:30:11.841893 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.841870    4524 generic.go:356] "Generic (PLEG): container finished" podID="714dacfe-8ec6-4cbc-8aed-534f8ad30f2f" containerID="1e96f43298e7f98fe1f771b738bacf87f88542e1bdad46f92c9866ba89e22128" exitCode=60
Oct 30 19:30:11.842007 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.841954    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf" event={"ID":"714dacfe-8ec6-4cbc-8aed-534f8ad30f2f","Type":"ContainerDied","Data":"1e96f43298e7f98fe1f771b738bacf87f88542e1bdad46f92c9866ba89e22128"}
Oct 30 19:30:11.842162 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.842144    4524 scope.go:122] "RemoveContainer" containerID="1e96f43298e7f98fe1f771b738bacf87f88542e1bdad46f92c9866ba89e22128"
Oct 30 19:30:11.845826 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.845810    4524 scope.go:122] "RemoveContainer" containerID="5426cab574c22fa1020164e69da8935f2819b0e710c84ae0ab0e7970cab71506"
Oct 30 19:30:11.846068 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:11.846041    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt_odh-ai-gateway-infra(c436fb87-4351-4a27-9286-216883df85cd)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt" podUID="c436fb87-4351-4a27-9286-216883df85cd"
Oct 30 19:30:11.847342 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.847320    4524 generic.go:356] "Generic (PLEG): container finished" podID="2d5c4eb6-06df-4544-84de-78b3c8b0f4e9" containerID="d7c15f5d043272b394114677117a0609d47b25f3d8c8cf1ea2fb577477fe6ac4" exitCode=60
Oct 30 19:30:11.847432 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.847404    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp" event={"ID":"2d5c4eb6-06df-4544-84de-78b3c8b0f4e9","Type":"ContainerDied","Data":"d7c15f5d043272b394114677117a0609d47b25f3d8c8cf1ea2fb577477fe6ac4"}
Oct 30 19:30:11.847571 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.847559    4524 scope.go:122] "RemoveContainer" containerID="d7c15f5d043272b394114677117a0609d47b25f3d8c8cf1ea2fb577477fe6ac4"
Oct 30 19:30:11.849257 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.849128    4524 scope.go:122] "RemoveContainer" containerID="b71879018815d86e282d8f0e69001291a1091babd9d025219863a4f39a8df76d"
Oct 30 19:30:11.850767 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:11.850496    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b71879018815d86e282d8f0e69001291a1091babd9d025219863a4f39a8df76d\": container with ID starting with b71879018815d86e282d8f0e69001291a1091babd9d025219863a4f39a8df76d not found: ID does not exist" containerID="b71879018815d86e282d8f0e69001291a1091babd9d025219863a4f39a8df76d"
Oct 30 19:30:11.850767 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.850525    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b71879018815d86e282d8f0e69001291a1091babd9d025219863a4f39a8df76d"} err="failed to get container status \"b71879018815d86e282d8f0e69001291a1091babd9d025219863a4f39a8df76d\": rpc error: code = NotFound desc = could not find container \"b71879018815d86e282d8f0e69001291a1091babd9d025219863a4f39a8df76d\": container with ID starting with b71879018815d86e282d8f0e69001291a1091babd9d025219863a4f39a8df76d not found: ID does not exist"
Oct 30 19:30:11.857937 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.857073    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85" event={"ID":"f17debbb-3dd6-47a2-8799-376d5735185c","Type":"ContainerStarted","Data":"90cbbe80b146957f6fd3be8c09573e235755b1bd691f350eb6e16913f7aef946"}
Oct 30 19:30:11.859792 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.859753    4524 generic.go:356] "Generic (PLEG): container finished" podID="a22cffda-4a00-4d56-a975-6ad1ab789847" containerID="bfc10ecdd231db5aaaca9c9160ee8e8a43316ca24edba23ac4ab9e10d40ca4c6" exitCode=60
Oct 30 19:30:11.859934 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.859916    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc" event={"ID":"a22cffda-4a00-4d56-a975-6ad1ab789847","Type":"ContainerDied","Data":"bfc10ecdd231db5aaaca9c9160ee8e8a43316ca24edba23ac4ab9e10d40ca4c6"}
Oct 30 19:30:11.860387 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.860365    4524 scope.go:122] "RemoveContainer" containerID="bfc10ecdd231db5aaaca9c9160ee8e8a43316ca24edba23ac4ab9e10d40ca4c6"
Oct 30 19:30:11.866761 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.866737    4524 generic.go:356] "Generic (PLEG): container finished" podID="555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" containerID="81d6076e7f972a1be08e4feaf13fb91095394764249adff8a8909f31913def37" exitCode=0
Oct 30 19:30:11.873749 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.866954    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"
Oct 30 19:30:11.875457 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.866972    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n" event={"ID":"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d","Type":"ContainerDied","Data":"81d6076e7f972a1be08e4feaf13fb91095394764249adff8a8909f31913def37"}
Oct 30 19:30:11.875457 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.874603    4524 scope.go:122] "RemoveContainer" containerID="81d6076e7f972a1be08e4feaf13fb91095394764249adff8a8909f31913def37"
Oct 30 19:30:11.877397 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.877374    4524 generic.go:356] "Generic (PLEG): container finished" podID="cc886570-4bd4-4f6d-8d49-2bcb11921b94" containerID="e9e21d4e476d3aca7bbdb5e19995e7ea39984cc1bb701e3e3c09806da21dcdd4" exitCode=0
Oct 30 19:30:11.877539 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.877501    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4" event={"ID":"cc886570-4bd4-4f6d-8d49-2bcb11921b94","Type":"ContainerDied","Data":"e9e21d4e476d3aca7bbdb5e19995e7ea39984cc1bb701e3e3c09806da21dcdd4"}
Oct 30 19:30:11.934312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.931450    4524 generic.go:356] "Generic (PLEG): container finished" podID="ce341e1c-1cbb-4ecc-8fda-c9088901b276" containerID="38adb2dee843852c3ec1566780493d53d5c8183a1d24b3d788d78299e689e0ff" exitCode=60
Oct 30 19:30:11.934312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.931564    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn" event={"ID":"ce341e1c-1cbb-4ecc-8fda-c9088901b276","Type":"ContainerDied","Data":"38adb2dee843852c3ec1566780493d53d5c8183a1d24b3d788d78299e689e0ff"}
Oct 30 19:30:11.934312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.932075    4524 scope.go:122] "RemoveContainer" containerID="38adb2dee843852c3ec1566780493d53d5c8183a1d24b3d788d78299e689e0ff"
Oct 30 19:30:11.955464 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.955435    4524 generic.go:356] "Generic (PLEG): container finished" podID="a4fd9180-f4aa-4986-b2f0-955296fe17ee" containerID="da7324c6301810498df84c172f6661cc8934360a23252f40bc67b5bd2a8d9ce3" exitCode=0
Oct 30 19:30:11.955557 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.955540    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk" event={"ID":"a4fd9180-f4aa-4986-b2f0-955296fe17ee","Type":"ContainerDied","Data":"da7324c6301810498df84c172f6661cc8934360a23252f40bc67b5bd2a8d9ce3"}
Oct 30 19:30:11.960524 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:11.960435    4524 scope.go:122] "RemoveContainer" containerID="76fd6a0f016f9aa7763f1bec2b8168659a42656eede582ee7d27d4462fd662e5"
Oct 30 19:30:11.961095 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:11.960752    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9_odh-ai-gateway-infra(652b5102-d21a-4673-b955-fb005179323e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9" podUID="652b5102-d21a-4673-b955-fb005179323e"
Oct 30 19:30:12.015473 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.011155    4524 scope.go:122] "RemoveContainer" containerID="c0ab990afb10ae4463e957a9700ca91164e8f6a1ae6f81bb4fa0f5557159e3a3"
Oct 30 19:30:12.015473 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:12.011676    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp_odh-ai-gateway-infra(7543e764-a757-4962-8342-f6d9d9e0f368)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp" podUID="7543e764-a757-4962-8342-f6d9d9e0f368"
Oct 30 19:30:12.015473 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.014458    4524 generic.go:356] "Generic (PLEG): container finished" podID="27521cd5-2e0a-4836-af74-6a96ea61ba6d" containerID="7890fce46c36779fa0928563db875aa4d7eef9aad1a51b32d47a83bd9c57d544" exitCode=60
Oct 30 19:30:12.015473 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.015278    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl" event={"ID":"27521cd5-2e0a-4836-af74-6a96ea61ba6d","Type":"ContainerDied","Data":"7890fce46c36779fa0928563db875aa4d7eef9aad1a51b32d47a83bd9c57d544"}
Oct 30 19:30:12.015726 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.015573    4524 scope.go:122] "RemoveContainer" containerID="7890fce46c36779fa0928563db875aa4d7eef9aad1a51b32d47a83bd9c57d544"
Oct 30 19:30:12.035456 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.035430    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-kube-api-access-x7v6r\" (UniqueName: \"kubernetes.io/projected/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-kube-api-access-x7v6r\") pod \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") "
Oct 30 19:30:12.035572 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.035523    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-workload-certs\") pod \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") "
Oct 30 19:30:12.035630 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.035609    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istiod-ca-cert\") pod \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") "
Oct 30 19:30:12.035680 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.035638    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-podinfo\") pod \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") "
Oct 30 19:30:12.036287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.035725    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-workload-socket\") pod \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") "
Oct 30 19:30:12.036287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.035795    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-envoy\") pod \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") "
Oct 30 19:30:12.036287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.035822    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-token\" (UniqueName: \"kubernetes.io/projected/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-token\") pod \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") "
Oct 30 19:30:12.036287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.035851    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-credential-socket\") pod \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") "
Oct 30 19:30:12.036287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.035913    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-data\") pod \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\" (UID: \"555d6782-ab13-48bd-8ef5-73ebe1cfcf1d\") "
Oct 30 19:30:12.036287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.035950    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-workload-certs" pod "555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" (UID: "555d6782-ab13-48bd-8ef5-73ebe1cfcf1d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:12.037042 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.036885    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:12.037533 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.037325    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istiod-ca-cert" pod "555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" (UID: "555d6782-ab13-48bd-8ef5-73ebe1cfcf1d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:12.039228 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.039097    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-kube-api-access-x7v6r" pod "555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" (UID: "555d6782-ab13-48bd-8ef5-73ebe1cfcf1d"). InnerVolumeSpecName "kube-api-access-x7v6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:12.039228 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.039195    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-credential-socket" pod "555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" (UID: "555d6782-ab13-48bd-8ef5-73ebe1cfcf1d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:12.039970 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.039926    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-data" pod "555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" (UID: "555d6782-ab13-48bd-8ef5-73ebe1cfcf1d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:12.039970 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.039950    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-workload-socket" pod "555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" (UID: "555d6782-ab13-48bd-8ef5-73ebe1cfcf1d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:12.043381 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.043347    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-envoy" pod "555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" (UID: "555d6782-ab13-48bd-8ef5-73ebe1cfcf1d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:12.043469 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.043414    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-token" pod "555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" (UID: "555d6782-ab13-48bd-8ef5-73ebe1cfcf1d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:12.043531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.043512    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-podinfo" pod "555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" (UID: "555d6782-ab13-48bd-8ef5-73ebe1cfcf1d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:30:12.063272 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.063233    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85" podStartSLOduration=12.06321926 podStartE2EDuration="12.06321926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:12.058110693 +0000 UTC m=+1443.799101771" watchObservedRunningTime="2026-10-30 19:30:12.06321926 +0000 UTC m=+1443.804210354"
Oct 30 19:30:12.087485 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.087459    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"]
Oct 30 19:30:12.095054 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.095032    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-zmtd9"]
Oct 30 19:30:12.139830 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.138950    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:12.139830 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.138977    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:12.139830 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.138994    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:12.139830 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.139009    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:12.139830 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.139026    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7v6r\" (UniqueName: \"kubernetes.io/projected/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-kube-api-access-x7v6r\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:12.139830 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.139042    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:12.139830 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.139056    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:12.139830 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.139071    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:12.221049 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.221023    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"]
Oct 30 19:30:12.226282 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.226257    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-kcc4n"]
Oct 30 19:30:12.486456 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.484564    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" path="/var/lib/kubelet/pods/555d6782-ab13-48bd-8ef5-73ebe1cfcf1d/volumes"
Oct 30 19:30:12.487689 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.487666    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="83f517e5-9b8e-425a-845b-56253704cd78" path="/var/lib/kubelet/pods/83f517e5-9b8e-425a-845b-56253704cd78/volumes"
Oct 30 19:30:12.493652 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.493372    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:30:12.648593 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.648558    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-workload-socket\") pod \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") "
Oct 30 19:30:12.649084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.648632    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4fd9180-f4aa-4986-b2f0-955296fe17ee-kube-api-access-2w8nh\" (UniqueName: \"kubernetes.io/projected/a4fd9180-f4aa-4986-b2f0-955296fe17ee-kube-api-access-2w8nh\") pod \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") "
Oct 30 19:30:12.649084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.648690    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-workload-certs\") pod \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") "
Oct 30 19:30:12.649084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.648744    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-envoy\") pod \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") "
Oct 30 19:30:12.649084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.648850    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istiod-ca-cert\") pod \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") "
Oct 30 19:30:12.649084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.648883    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-workload-socket" pod "a4fd9180-f4aa-4986-b2f0-955296fe17ee" (UID: "a4fd9180-f4aa-4986-b2f0-955296fe17ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:12.649084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.648933    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-data\") pod \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") "
Oct 30 19:30:12.649084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.648992    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-workload-certs" pod "a4fd9180-f4aa-4986-b2f0-955296fe17ee" (UID: "a4fd9180-f4aa-4986-b2f0-955296fe17ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:12.649084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.649055    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-podinfo\") pod \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") "
Oct 30 19:30:12.649506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.649095    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-credential-socket\") pod \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") "
Oct 30 19:30:12.649506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.649132    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-token\" (UniqueName: \"kubernetes.io/projected/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-token\") pod \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\" (UID: \"a4fd9180-f4aa-4986-b2f0-955296fe17ee\") "
Oct 30 19:30:12.649506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.649329    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-data" pod "a4fd9180-f4aa-4986-b2f0-955296fe17ee" (UID: "a4fd9180-f4aa-4986-b2f0-955296fe17ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:12.649506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.649372    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istiod-ca-cert" pod "a4fd9180-f4aa-4986-b2f0-955296fe17ee" (UID: "a4fd9180-f4aa-4986-b2f0-955296fe17ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:12.649506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.649397    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-credential-socket" pod "a4fd9180-f4aa-4986-b2f0-955296fe17ee" (UID: "a4fd9180-f4aa-4986-b2f0-955296fe17ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:12.649939 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.649918    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:12.650037 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.649942    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:12.650037 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.649961    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:12.650037 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.649976    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:12.650037 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.649989    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:12.668657 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.668617    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-envoy" pod "a4fd9180-f4aa-4986-b2f0-955296fe17ee" (UID: "a4fd9180-f4aa-4986-b2f0-955296fe17ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:30:12.668775 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.668717    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-token" pod "a4fd9180-f4aa-4986-b2f0-955296fe17ee" (UID: "a4fd9180-f4aa-4986-b2f0-955296fe17ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:12.668775 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.668768    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-podinfo" pod "a4fd9180-f4aa-4986-b2f0-955296fe17ee" (UID: "a4fd9180-f4aa-4986-b2f0-955296fe17ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:30:12.671419 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.671374    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4fd9180-f4aa-4986-b2f0-955296fe17ee-kube-api-access-2w8nh" pod "a4fd9180-f4aa-4986-b2f0-955296fe17ee" (UID: "a4fd9180-f4aa-4986-b2f0-955296fe17ee"). InnerVolumeSpecName "kube-api-access-2w8nh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:12.755293 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.753024    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:12.755293 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.753065    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:12.755293 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.753081    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2w8nh\" (UniqueName: \"kubernetes.io/projected/a4fd9180-f4aa-4986-b2f0-955296fe17ee-kube-api-access-2w8nh\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:12.755293 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:12.753096    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4fd9180-f4aa-4986-b2f0-955296fe17ee-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:13.029986 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.029901    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw" event={"ID":"60537a63-2e4e-4441-a28e-2389fe0d0e93","Type":"ContainerStarted","Data":"c1e1c7c447c9188cde79b46a9ba68042ed8cd1f37e48d2f74a2e23852d08a1c4"}
Oct 30 19:30:13.031967 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.031932    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf" event={"ID":"714dacfe-8ec6-4cbc-8aed-534f8ad30f2f","Type":"ContainerStarted","Data":"895970cf4f7269eae95e75ba25085c824d679e0a05821fb01d0bff74e133bdb3"}
Oct 30 19:30:13.033989 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.033967    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp" event={"ID":"2d5c4eb6-06df-4544-84de-78b3c8b0f4e9","Type":"ContainerStarted","Data":"4cc47c9e3eebf5bf24c109f05e145571eee752fff8c039179b66f14f0f17dacc"}
Oct 30 19:30:13.035838 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.035816    4524 generic.go:356] "Generic (PLEG): container finished" podID="f17debbb-3dd6-47a2-8799-376d5735185c" containerID="90cbbe80b146957f6fd3be8c09573e235755b1bd691f350eb6e16913f7aef946" exitCode=60
Oct 30 19:30:13.035926 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.035861    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85" event={"ID":"f17debbb-3dd6-47a2-8799-376d5735185c","Type":"ContainerDied","Data":"90cbbe80b146957f6fd3be8c09573e235755b1bd691f350eb6e16913f7aef946"}
Oct 30 19:30:13.035926 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.035912    4524 scope.go:122] "RemoveContainer" containerID="1604c39c1469d7a0497bd7d21fccdd118c8ce10ba39e751df8dab248576bd7c2"
Oct 30 19:30:13.036204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.036142    4524 scope.go:122] "RemoveContainer" containerID="90cbbe80b146957f6fd3be8c09573e235755b1bd691f350eb6e16913f7aef946"
Oct 30 19:30:13.036563 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:13.036540    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85_odh-ai-gateway-infra(f17debbb-3dd6-47a2-8799-376d5735185c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85" podUID="f17debbb-3dd6-47a2-8799-376d5735185c"
Oct 30 19:30:13.038074 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.038048    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc" event={"ID":"a22cffda-4a00-4d56-a975-6ad1ab789847","Type":"ContainerStarted","Data":"6497fc14a9e4f09a650abe6da015850fb53f42c5584a343d3f7452834ff0f020"}
Oct 30 19:30:13.041023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.041005    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn" event={"ID":"ce341e1c-1cbb-4ecc-8fda-c9088901b276","Type":"ContainerStarted","Data":"a6d3c3c02103ea9a63d273f6d47b5805a0e0763c4cdc3afda6f9366a174142bf"}
Oct 30 19:30:13.042439 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.042423    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"
Oct 30 19:30:13.042513 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.042434    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk" event={"ID":"a4fd9180-f4aa-4986-b2f0-955296fe17ee","Type":"ContainerDied","Data":"d5fe525b875c63d46ff2d2b1030993046ccdad40c4a41bca3567ae31f92d71b2"}
Oct 30 19:30:13.044584 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.044564    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl" event={"ID":"27521cd5-2e0a-4836-af74-6a96ea61ba6d","Type":"ContainerStarted","Data":"fe3760c03a09db92a4c23b471230d7e8ae4a88f4b12f16b3654401709425669d"}
Oct 30 19:30:13.051338 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.051319    4524 scope.go:122] "RemoveContainer" containerID="da7324c6301810498df84c172f6661cc8934360a23252f40bc67b5bd2a8d9ce3"
Oct 30 19:30:13.292566 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.292401    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"]
Oct 30 19:30:13.412612 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.412574    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-9lxmk"]
Oct 30 19:30:13.635296 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.635268    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4"
Oct 30 19:30:13.703055 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.703010    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cc886570-4bd4-4f6d-8d49-2bcb11921b94-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc886570-4bd4-4f6d-8d49-2bcb11921b94-maas-api-openshift-ca\") pod \"cc886570-4bd4-4f6d-8d49-2bcb11921b94\" (UID: \"cc886570-4bd4-4f6d-8d49-2bcb11921b94\") "
Oct 30 19:30:13.703679 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.703655    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc886570-4bd4-4f6d-8d49-2bcb11921b94-kube-api-access-cxcrr\" (UniqueName: \"kubernetes.io/projected/cc886570-4bd4-4f6d-8d49-2bcb11921b94-kube-api-access-cxcrr\") pod \"cc886570-4bd4-4f6d-8d49-2bcb11921b94\" (UID: \"cc886570-4bd4-4f6d-8d49-2bcb11921b94\") "
Oct 30 19:30:13.703879 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.703858    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cc886570-4bd4-4f6d-8d49-2bcb11921b94-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc886570-4bd4-4f6d-8d49-2bcb11921b94-maas-api-xks-ca\") pod \"cc886570-4bd4-4f6d-8d49-2bcb11921b94\" (UID: \"cc886570-4bd4-4f6d-8d49-2bcb11921b94\") "
Oct 30 19:30:13.704127 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.703880    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc886570-4bd4-4f6d-8d49-2bcb11921b94-maas-api-openshift-ca" pod "cc886570-4bd4-4f6d-8d49-2bcb11921b94" (UID: "cc886570-4bd4-4f6d-8d49-2bcb11921b94"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:13.704862 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.704838    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc886570-4bd4-4f6d-8d49-2bcb11921b94-maas-api-openshift-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:13.706360 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.706304    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc886570-4bd4-4f6d-8d49-2bcb11921b94-maas-api-xks-ca" pod "cc886570-4bd4-4f6d-8d49-2bcb11921b94" (UID: "cc886570-4bd4-4f6d-8d49-2bcb11921b94"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:30:13.706552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.706528    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc886570-4bd4-4f6d-8d49-2bcb11921b94-kube-api-access-cxcrr" pod "cc886570-4bd4-4f6d-8d49-2bcb11921b94" (UID: "cc886570-4bd4-4f6d-8d49-2bcb11921b94"). InnerVolumeSpecName "kube-api-access-cxcrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:13.840263 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.819934    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxcrr\" (UniqueName: \"kubernetes.io/projected/cc886570-4bd4-4f6d-8d49-2bcb11921b94-kube-api-access-cxcrr\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:13.840263 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:13.819969    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc886570-4bd4-4f6d-8d49-2bcb11921b94-maas-api-xks-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:14.055785 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.055746    4524 generic.go:356] "Generic (PLEG): container finished" podID="a22cffda-4a00-4d56-a975-6ad1ab789847" containerID="6497fc14a9e4f09a650abe6da015850fb53f42c5584a343d3f7452834ff0f020" exitCode=60
Oct 30 19:30:14.055958 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.055839    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc" event={"ID":"a22cffda-4a00-4d56-a975-6ad1ab789847","Type":"ContainerDied","Data":"6497fc14a9e4f09a650abe6da015850fb53f42c5584a343d3f7452834ff0f020"}
Oct 30 19:30:14.055958 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.055924    4524 scope.go:122] "RemoveContainer" containerID="bfc10ecdd231db5aaaca9c9160ee8e8a43316ca24edba23ac4ab9e10d40ca4c6"
Oct 30 19:30:14.056267 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.056249    4524 scope.go:122] "RemoveContainer" containerID="6497fc14a9e4f09a650abe6da015850fb53f42c5584a343d3f7452834ff0f020"
Oct 30 19:30:14.056626 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:14.056602    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc_odh-ai-gateway-infra(a22cffda-4a00-4d56-a975-6ad1ab789847)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc" podUID="a22cffda-4a00-4d56-a975-6ad1ab789847"
Oct 30 19:30:14.057427 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.057413    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4"
Oct 30 19:30:14.057502 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.057444    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4" event={"ID":"cc886570-4bd4-4f6d-8d49-2bcb11921b94","Type":"ContainerDied","Data":"f37e850becd9eebbf2ed67607eee46a2166a6157f8d7aae37890427c0a089921"}
Oct 30 19:30:14.057502 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.057483    4524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f37e850becd9eebbf2ed67607eee46a2166a6157f8d7aae37890427c0a089921"
Oct 30 19:30:14.059351 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.059329    4524 generic.go:356] "Generic (PLEG): container finished" podID="ce341e1c-1cbb-4ecc-8fda-c9088901b276" containerID="a6d3c3c02103ea9a63d273f6d47b5805a0e0763c4cdc3afda6f9366a174142bf" exitCode=60
Oct 30 19:30:14.059450 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.059397    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn" event={"ID":"ce341e1c-1cbb-4ecc-8fda-c9088901b276","Type":"ContainerDied","Data":"a6d3c3c02103ea9a63d273f6d47b5805a0e0763c4cdc3afda6f9366a174142bf"}
Oct 30 19:30:14.059723 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.059706    4524 scope.go:122] "RemoveContainer" containerID="a6d3c3c02103ea9a63d273f6d47b5805a0e0763c4cdc3afda6f9366a174142bf"
Oct 30 19:30:14.060026 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:14.060006    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn_odh-ai-gateway-infra(ce341e1c-1cbb-4ecc-8fda-c9088901b276)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn" podUID="ce341e1c-1cbb-4ecc-8fda-c9088901b276"
Oct 30 19:30:14.062738 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.062714    4524 generic.go:356] "Generic (PLEG): container finished" podID="27521cd5-2e0a-4836-af74-6a96ea61ba6d" containerID="fe3760c03a09db92a4c23b471230d7e8ae4a88f4b12f16b3654401709425669d" exitCode=60
Oct 30 19:30:14.062842 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.062800    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl" event={"ID":"27521cd5-2e0a-4836-af74-6a96ea61ba6d","Type":"ContainerDied","Data":"fe3760c03a09db92a4c23b471230d7e8ae4a88f4b12f16b3654401709425669d"}
Oct 30 19:30:14.063035 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.063018    4524 scope.go:122] "RemoveContainer" containerID="fe3760c03a09db92a4c23b471230d7e8ae4a88f4b12f16b3654401709425669d"
Oct 30 19:30:14.063353 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:14.063320    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl_odh-ai-gateway-infra(27521cd5-2e0a-4836-af74-6a96ea61ba6d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl" podUID="27521cd5-2e0a-4836-af74-6a96ea61ba6d"
Oct 30 19:30:14.064973 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.064954    4524 generic.go:356] "Generic (PLEG): container finished" podID="60537a63-2e4e-4441-a28e-2389fe0d0e93" containerID="c1e1c7c447c9188cde79b46a9ba68042ed8cd1f37e48d2f74a2e23852d08a1c4" exitCode=7
Oct 30 19:30:14.065106 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.065022    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw" event={"ID":"60537a63-2e4e-4441-a28e-2389fe0d0e93","Type":"ContainerDied","Data":"c1e1c7c447c9188cde79b46a9ba68042ed8cd1f37e48d2f74a2e23852d08a1c4"}
Oct 30 19:30:14.065254 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.065233    4524 scope.go:122] "RemoveContainer" containerID="c1e1c7c447c9188cde79b46a9ba68042ed8cd1f37e48d2f74a2e23852d08a1c4"
Oct 30 19:30:14.065552 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:14.065519    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw_odh-ai-gateway-infra(60537a63-2e4e-4441-a28e-2389fe0d0e93)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw" podUID="60537a63-2e4e-4441-a28e-2389fe0d0e93"
Oct 30 19:30:14.068639 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.068614    4524 generic.go:356] "Generic (PLEG): container finished" podID="714dacfe-8ec6-4cbc-8aed-534f8ad30f2f" containerID="895970cf4f7269eae95e75ba25085c824d679e0a05821fb01d0bff74e133bdb3" exitCode=60
Oct 30 19:30:14.068743 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.068729    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf" event={"ID":"714dacfe-8ec6-4cbc-8aed-534f8ad30f2f","Type":"ContainerDied","Data":"895970cf4f7269eae95e75ba25085c824d679e0a05821fb01d0bff74e133bdb3"}
Oct 30 19:30:14.069060 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.069043    4524 scope.go:122] "RemoveContainer" containerID="895970cf4f7269eae95e75ba25085c824d679e0a05821fb01d0bff74e133bdb3"
Oct 30 19:30:14.069720 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:14.069309    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf_odh-ai-gateway-infra(714dacfe-8ec6-4cbc-8aed-534f8ad30f2f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf" podUID="714dacfe-8ec6-4cbc-8aed-534f8ad30f2f"
Oct 30 19:30:14.071704 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.071685    4524 generic.go:356] "Generic (PLEG): container finished" podID="2d5c4eb6-06df-4544-84de-78b3c8b0f4e9" containerID="4cc47c9e3eebf5bf24c109f05e145571eee752fff8c039179b66f14f0f17dacc" exitCode=60
Oct 30 19:30:14.071798 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.071755    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp" event={"ID":"2d5c4eb6-06df-4544-84de-78b3c8b0f4e9","Type":"ContainerDied","Data":"4cc47c9e3eebf5bf24c109f05e145571eee752fff8c039179b66f14f0f17dacc"}
Oct 30 19:30:14.072043 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.072021    4524 scope.go:122] "RemoveContainer" containerID="4cc47c9e3eebf5bf24c109f05e145571eee752fff8c039179b66f14f0f17dacc"
Oct 30 19:30:14.072349 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:14.072325    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp_odh-ai-gateway-infra(2d5c4eb6-06df-4544-84de-78b3c8b0f4e9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp" podUID="2d5c4eb6-06df-4544-84de-78b3c8b0f4e9"
Oct 30 19:30:14.074448 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.074431    4524 scope.go:122] "RemoveContainer" containerID="90cbbe80b146957f6fd3be8c09573e235755b1bd691f350eb6e16913f7aef946"
Oct 30 19:30:14.074698 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:14.074677    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85_odh-ai-gateway-infra(f17debbb-3dd6-47a2-8799-376d5735185c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85" podUID="f17debbb-3dd6-47a2-8799-376d5735185c"
Oct 30 19:30:14.075196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.075180    4524 scope.go:122] "RemoveContainer" containerID="38adb2dee843852c3ec1566780493d53d5c8183a1d24b3d788d78299e689e0ff"
Oct 30 19:30:14.091905 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.091886    4524 scope.go:122] "RemoveContainer" containerID="7890fce46c36779fa0928563db875aa4d7eef9aad1a51b32d47a83bd9c57d544"
Oct 30 19:30:14.113511 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.113479    4524 scope.go:122] "RemoveContainer" containerID="b175cd4be4e22c4c2bfc9e12e1c87008610fa28a6fa16d46a6a48940db1bedf1"
Oct 30 19:30:14.129362 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.129342    4524 scope.go:122] "RemoveContainer" containerID="1e96f43298e7f98fe1f771b738bacf87f88542e1bdad46f92c9866ba89e22128"
Oct 30 19:30:14.144496 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.144477    4524 scope.go:122] "RemoveContainer" containerID="d7c15f5d043272b394114677117a0609d47b25f3d8c8cf1ea2fb577477fe6ac4"
Oct 30 19:30:14.398109 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:14.398023    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a4fd9180-f4aa-4986-b2f0-955296fe17ee" path="/var/lib/kubelet/pods/a4fd9180-f4aa-4986-b2f0-955296fe17ee/volumes"
Oct 30 19:30:15.086915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:15.086882    4524 scope.go:122] "RemoveContainer" containerID="a6d3c3c02103ea9a63d273f6d47b5805a0e0763c4cdc3afda6f9366a174142bf"
Oct 30 19:30:15.087385 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:15.087339    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn_odh-ai-gateway-infra(ce341e1c-1cbb-4ecc-8fda-c9088901b276)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn" podUID="ce341e1c-1cbb-4ecc-8fda-c9088901b276"
Oct 30 19:30:15.088820 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:15.088798    4524 scope.go:122] "RemoveContainer" containerID="fe3760c03a09db92a4c23b471230d7e8ae4a88f4b12f16b3654401709425669d"
Oct 30 19:30:15.089127 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:15.089105    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl_odh-ai-gateway-infra(27521cd5-2e0a-4836-af74-6a96ea61ba6d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl" podUID="27521cd5-2e0a-4836-af74-6a96ea61ba6d"
Oct 30 19:30:15.091678 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:15.091657    4524 scope.go:122] "RemoveContainer" containerID="c1e1c7c447c9188cde79b46a9ba68042ed8cd1f37e48d2f74a2e23852d08a1c4"
Oct 30 19:30:15.092298 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:15.092273    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw_odh-ai-gateway-infra(60537a63-2e4e-4441-a28e-2389fe0d0e93)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw" podUID="60537a63-2e4e-4441-a28e-2389fe0d0e93"
Oct 30 19:30:15.094031 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:15.094012    4524 scope.go:122] "RemoveContainer" containerID="895970cf4f7269eae95e75ba25085c824d679e0a05821fb01d0bff74e133bdb3"
Oct 30 19:30:15.094381 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:15.094353    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf_odh-ai-gateway-infra(714dacfe-8ec6-4cbc-8aed-534f8ad30f2f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf" podUID="714dacfe-8ec6-4cbc-8aed-534f8ad30f2f"
Oct 30 19:30:15.095924 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:15.095906    4524 scope.go:122] "RemoveContainer" containerID="4cc47c9e3eebf5bf24c109f05e145571eee752fff8c039179b66f14f0f17dacc"
Oct 30 19:30:15.096240 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:15.096166    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp_odh-ai-gateway-infra(2d5c4eb6-06df-4544-84de-78b3c8b0f4e9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp" podUID="2d5c4eb6-06df-4544-84de-78b3c8b0f4e9"
Oct 30 19:30:15.097998 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:15.097974    4524 scope.go:122] "RemoveContainer" containerID="6497fc14a9e4f09a650abe6da015850fb53f42c5584a343d3f7452834ff0f020"
Oct 30 19:30:15.098249 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:15.098225    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc_odh-ai-gateway-infra(a22cffda-4a00-4d56-a975-6ad1ab789847)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc" podUID="a22cffda-4a00-4d56-a975-6ad1ab789847"
Oct 30 19:30:15.732065 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:15.732025    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:30:22.018158 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.018122    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q"
Oct 30 19:30:22.399195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.398836    4524 scope.go:122] "RemoveContainer" containerID="76fd6a0f016f9aa7763f1bec2b8168659a42656eede582ee7d27d4462fd662e5"
Oct 30 19:30:22.400943 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.399958    4524 scope.go:122] "RemoveContainer" containerID="5426cab574c22fa1020164e69da8935f2819b0e710c84ae0ab0e7970cab71506"
Oct 30 19:30:22.437313 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.437051    4524 scope.go:122] "RemoveContainer" containerID="7dbfc3f919fdd1f49433d8069ce99555d7ba4b72902ce9e191ad651d0824f682"
Oct 30 19:30:22.597732 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.577193    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"]
Oct 30 19:30:22.598562 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.598529    4524 cpu_manager.go:414] "containerMap: removing container" podUID="555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" containerName="istio-proxy"
Oct 30 19:30:22.598562 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.598558    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" containerName="istio-proxy"
Oct 30 19:30:22.598772 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.598572    4524 cpu_manager.go:414] "containerMap: removing container" podUID="83f517e5-9b8e-425a-845b-56253704cd78" containerName="istio-proxy"
Oct 30 19:30:22.598772 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.598580    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="83f517e5-9b8e-425a-845b-56253704cd78" containerName="istio-proxy"
Oct 30 19:30:22.598772 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.598597    4524 cpu_manager.go:414] "containerMap: removing container" podUID="cc886570-4bd4-4f6d-8d49-2bcb11921b94" containerName="revoke-keys"
Oct 30 19:30:22.598772 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.598604    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc886570-4bd4-4f6d-8d49-2bcb11921b94" containerName="revoke-keys"
Oct 30 19:30:22.598772 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.598633    4524 cpu_manager.go:414] "containerMap: removing container" podUID="a4fd9180-f4aa-4986-b2f0-955296fe17ee" containerName="istio-proxy"
Oct 30 19:30:22.598772 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.598640    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4fd9180-f4aa-4986-b2f0-955296fe17ee" containerName="istio-proxy"
Oct 30 19:30:22.599105 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.598883    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="555d6782-ab13-48bd-8ef5-73ebe1cfcf1d" containerName="istio-proxy"
Oct 30 19:30:22.599105 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.598897    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="83f517e5-9b8e-425a-845b-56253704cd78" containerName="istio-proxy"
Oct 30 19:30:22.599105 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.598907    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc886570-4bd4-4f6d-8d49-2bcb11921b94" containerName="revoke-keys"
Oct 30 19:30:22.599105 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.598921    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="a4fd9180-f4aa-4986-b2f0-955296fe17ee" containerName="istio-proxy"
Oct 30 19:30:22.605599 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.605108    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"]
Oct 30 19:30:22.605599 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.605235    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"
Oct 30 19:30:22.609531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.609509    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-ef5dca-serving-cert\""
Oct 30 19:30:22.662635 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.662585    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh\" (UID: \"3108bb35-3f80-4f5b-b9a4-505982b1f1ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"
Oct 30 19:30:22.662635 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.662619    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sch5l\" (UniqueName: \"kubernetes.io/projected/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-kube-api-access-sch5l\") pod \"maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh\" (UID: \"3108bb35-3f80-4f5b-b9a4-505982b1f1ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"
Oct 30 19:30:22.662795 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.662774    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-maas-api-tls\") pod \"maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh\" (UID: \"3108bb35-3f80-4f5b-b9a4-505982b1f1ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"
Oct 30 19:30:22.662854 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.662818    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-metrics-certs\") pod \"maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh\" (UID: \"3108bb35-3f80-4f5b-b9a4-505982b1f1ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"
Oct 30 19:30:22.764882 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.764845    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-maas-api-tls\") pod \"maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh\" (UID: \"3108bb35-3f80-4f5b-b9a4-505982b1f1ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"
Oct 30 19:30:22.765058 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.764942    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-metrics-certs\") pod \"maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh\" (UID: \"3108bb35-3f80-4f5b-b9a4-505982b1f1ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"
Oct 30 19:30:22.765721 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.765692    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh\" (UID: \"3108bb35-3f80-4f5b-b9a4-505982b1f1ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"
Oct 30 19:30:22.765860 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.765746    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sch5l\" (UniqueName: \"kubernetes.io/projected/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-kube-api-access-sch5l\") pod \"maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh\" (UID: \"3108bb35-3f80-4f5b-b9a4-505982b1f1ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"
Oct 30 19:30:22.766876 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.766852    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh\" (UID: \"3108bb35-3f80-4f5b-b9a4-505982b1f1ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"
Oct 30 19:30:22.805909 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.805818    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-metrics-certs\") pod \"maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh\" (UID: \"3108bb35-3f80-4f5b-b9a4-505982b1f1ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"
Oct 30 19:30:22.872821 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.872789    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sch5l\" (UniqueName: \"kubernetes.io/projected/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-kube-api-access-sch5l\") pod \"maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh\" (UID: \"3108bb35-3f80-4f5b-b9a4-505982b1f1ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"
Oct 30 19:30:22.872975 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.872909    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-maas-api-tls\") pod \"maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh\" (UID: \"3108bb35-3f80-4f5b-b9a4-505982b1f1ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"
Oct 30 19:30:22.934268 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:22.934240    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"
Oct 30 19:30:23.402192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:23.395967    4524 scope.go:122] "RemoveContainer" containerID="c0ab990afb10ae4463e957a9700ca91164e8f6a1ae6f81bb4fa0f5557159e3a3"
Oct 30 19:30:23.585561 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:23.585495    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"]
Oct 30 19:30:23.673246 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:30:23.673147    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3108bb35_3f80_4f5b_b9a4_505982b1f1ee.slice/crio-faba862a06ab7a2c26d6bdd0f1be199b99a96385d7a59d66147c8d8dfdc2b10a WatchSource:0}: Error finding container faba862a06ab7a2c26d6bdd0f1be199b99a96385d7a59d66147c8d8dfdc2b10a: Status 404 returned error can't find the container with id faba862a06ab7a2c26d6bdd0f1be199b99a96385d7a59d66147c8d8dfdc2b10a
Oct 30 19:30:24.365879 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:24.365847    4524 generic.go:356] "Generic (PLEG): container finished" podID="3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d" containerID="e08893ad43ec0af8506bc1c2ac4964b1e778851027f1a7f3906d047e9e26cc19" exitCode=60
Oct 30 19:30:24.366059 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:24.365921    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x" event={"ID":"3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d","Type":"ContainerDied","Data":"e08893ad43ec0af8506bc1c2ac4964b1e778851027f1a7f3906d047e9e26cc19"}
Oct 30 19:30:24.366059 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:24.366008    4524 scope.go:122] "RemoveContainer" containerID="7dbfc3f919fdd1f49433d8069ce99555d7ba4b72902ce9e191ad651d0824f682"
Oct 30 19:30:24.366448 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:24.366429    4524 scope.go:122] "RemoveContainer" containerID="e08893ad43ec0af8506bc1c2ac4964b1e778851027f1a7f3906d047e9e26cc19"
Oct 30 19:30:24.366812 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:24.366785    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x_odh-ai-gateway-infra(3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x" podUID="3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d"
Oct 30 19:30:24.370225 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:24.370197    4524 generic.go:356] "Generic (PLEG): container finished" podID="c436fb87-4351-4a27-9286-216883df85cd" containerID="6b4b48b0d6a82d5d8e609f35338d7050192deefbf3a10ed2adbe065fdf154afa" exitCode=60
Oct 30 19:30:24.370457 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:24.370432    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt" event={"ID":"c436fb87-4351-4a27-9286-216883df85cd","Type":"ContainerDied","Data":"6b4b48b0d6a82d5d8e609f35338d7050192deefbf3a10ed2adbe065fdf154afa"}
Oct 30 19:30:24.370890 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:24.370869    4524 scope.go:122] "RemoveContainer" containerID="6b4b48b0d6a82d5d8e609f35338d7050192deefbf3a10ed2adbe065fdf154afa"
Oct 30 19:30:24.371333 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:24.371308    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt_odh-ai-gateway-infra(c436fb87-4351-4a27-9286-216883df85cd)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt" podUID="c436fb87-4351-4a27-9286-216883df85cd"
Oct 30 19:30:24.372674 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:24.372645    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh" event={"ID":"3108bb35-3f80-4f5b-b9a4-505982b1f1ee","Type":"ContainerStarted","Data":"a54040016ab4f8da8c7b4919c5e742698bbe973cfa7d663e4e2f98a4a709c073"}
Oct 30 19:30:24.372776 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:24.372699    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh" event={"ID":"3108bb35-3f80-4f5b-b9a4-505982b1f1ee","Type":"ContainerStarted","Data":"faba862a06ab7a2c26d6bdd0f1be199b99a96385d7a59d66147c8d8dfdc2b10a"}
Oct 30 19:30:24.372890 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:24.372868    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"
Oct 30 19:30:24.375739 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:24.375716    4524 generic.go:356] "Generic (PLEG): container finished" podID="652b5102-d21a-4673-b955-fb005179323e" containerID="c3f2ca69bdfbbd7b942e6d7ea0eb842e067ef6aa7301f63128209b34a61dbbfa" exitCode=60
Oct 30 19:30:24.375866 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:24.375788    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9" event={"ID":"652b5102-d21a-4673-b955-fb005179323e","Type":"ContainerDied","Data":"c3f2ca69bdfbbd7b942e6d7ea0eb842e067ef6aa7301f63128209b34a61dbbfa"}
Oct 30 19:30:24.376234 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:24.376214    4524 scope.go:122] "RemoveContainer" containerID="c3f2ca69bdfbbd7b942e6d7ea0eb842e067ef6aa7301f63128209b34a61dbbfa"
Oct 30 19:30:24.376656 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:24.376636    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9_odh-ai-gateway-infra(652b5102-d21a-4673-b955-fb005179323e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9" podUID="652b5102-d21a-4673-b955-fb005179323e"
Oct 30 19:30:24.378933 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:24.378909    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp" event={"ID":"7543e764-a757-4962-8342-f6d9d9e0f368","Type":"ContainerStarted","Data":"18078083379d92b2ef3a586f70688e90fa9859e176adde994a8568ff6d247ee7"}
Oct 30 19:30:24.384317 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:24.383433    4524 scope.go:122] "RemoveContainer" containerID="90cbbe80b146957f6fd3be8c09573e235755b1bd691f350eb6e16913f7aef946"
Oct 30 19:30:24.384317 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:24.383789    4524 scope.go:122] "RemoveContainer" containerID="895970cf4f7269eae95e75ba25085c824d679e0a05821fb01d0bff74e133bdb3"
Oct 30 19:30:24.384317 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:24.383905    4524 scope.go:122] "RemoveContainer" containerID="5426cab574c22fa1020164e69da8935f2819b0e710c84ae0ab0e7970cab71506"
Oct 30 19:30:24.408644 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:24.408621    4524 scope.go:122] "RemoveContainer" containerID="76fd6a0f016f9aa7763f1bec2b8168659a42656eede582ee7d27d4462fd662e5"
Oct 30 19:30:24.455610 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:24.455570    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh" podStartSLOduration=2.017743839 podStartE2EDuration="2.45555624s" totalImagesPullingTime="437.812401ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:24.451625984 +0000 UTC m=+1456.192617085" watchObservedRunningTime="2026-10-30 19:30:24.45555624 +0000 UTC m=+1456.196547318"
Oct 30 19:30:24.495636 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:24.495575    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp" podStartSLOduration=19.064345338 podStartE2EDuration="24.495556378s" totalImagesPullingTime="5.43121104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:24.490284179 +0000 UTC m=+1456.231275261" watchObservedRunningTime="2026-10-30 19:30:24.495556378 +0000 UTC m=+1456.236547457"
Oct 30 19:30:25.382859 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.382835    4524 scope.go:122] "RemoveContainer" containerID="fe3760c03a09db92a4c23b471230d7e8ae4a88f4b12f16b3654401709425669d"
Oct 30 19:30:25.383710 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.383684    4524 scope.go:122] "RemoveContainer" containerID="a6d3c3c02103ea9a63d273f6d47b5805a0e0763c4cdc3afda6f9366a174142bf"
Oct 30 19:30:25.396404 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.396378    4524 generic.go:356] "Generic (PLEG): container finished" podID="f17debbb-3dd6-47a2-8799-376d5735185c" containerID="dfefe5c860b0ee86a753273e0564e68153971ed1f7117345e4b9b6445db0d540" exitCode=60
Oct 30 19:30:25.396526 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.396457    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85" event={"ID":"f17debbb-3dd6-47a2-8799-376d5735185c","Type":"ContainerDied","Data":"dfefe5c860b0ee86a753273e0564e68153971ed1f7117345e4b9b6445db0d540"}
Oct 30 19:30:25.396596 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.396526    4524 scope.go:122] "RemoveContainer" containerID="90cbbe80b146957f6fd3be8c09573e235755b1bd691f350eb6e16913f7aef946"
Oct 30 19:30:25.396894 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.396873    4524 scope.go:122] "RemoveContainer" containerID="dfefe5c860b0ee86a753273e0564e68153971ed1f7117345e4b9b6445db0d540"
Oct 30 19:30:25.397132 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:25.397110    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85_odh-ai-gateway-infra(f17debbb-3dd6-47a2-8799-376d5735185c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85" podUID="f17debbb-3dd6-47a2-8799-376d5735185c"
Oct 30 19:30:25.411206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.410220    4524 generic.go:356] "Generic (PLEG): container finished" podID="7543e764-a757-4962-8342-f6d9d9e0f368" containerID="18078083379d92b2ef3a586f70688e90fa9859e176adde994a8568ff6d247ee7" exitCode=60
Oct 30 19:30:25.411206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.410343    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp" event={"ID":"7543e764-a757-4962-8342-f6d9d9e0f368","Type":"ContainerDied","Data":"18078083379d92b2ef3a586f70688e90fa9859e176adde994a8568ff6d247ee7"}
Oct 30 19:30:25.411206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.410820    4524 scope.go:122] "RemoveContainer" containerID="18078083379d92b2ef3a586f70688e90fa9859e176adde994a8568ff6d247ee7"
Oct 30 19:30:25.412582 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:25.412534    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp_odh-ai-gateway-infra(7543e764-a757-4962-8342-f6d9d9e0f368)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp" podUID="7543e764-a757-4962-8342-f6d9d9e0f368"
Oct 30 19:30:25.419997 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.419666    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x"]
Oct 30 19:30:25.421639 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.421602    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf" event={"ID":"714dacfe-8ec6-4cbc-8aed-534f8ad30f2f","Type":"ContainerStarted","Data":"0fe67f2ba2699aecb9f08e09924e11a8b495465f3ead55cbaac30e47e40952e8"}
Oct 30 19:30:25.438089 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.438061    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9"]
Oct 30 19:30:25.460397 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.460272    4524 scope.go:122] "RemoveContainer" containerID="c0ab990afb10ae4463e957a9700ca91164e8f6a1ae6f81bb4fa0f5557159e3a3"
Oct 30 19:30:25.512034 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.510810    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt"]
Oct 30 19:30:25.543398 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.538599    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp"]
Oct 30 19:30:25.881059 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.881027    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x"
Oct 30 19:30:25.918725 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.918690    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d-kube-api-access-m29h9\" (UniqueName: \"kubernetes.io/projected/3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d-kube-api-access-m29h9\") pod \"3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d\" (UID: \"3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d\") "
Oct 30 19:30:25.919210 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.919185    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d-maas-api-service-ca\") pod \"3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d\" (UID: \"3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d\") "
Oct 30 19:30:25.928759 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.921904    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d-maas-api-service-ca" pod "3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d" (UID: "3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:25.928759 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.924825    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d-kube-api-access-m29h9" pod "3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d" (UID: "3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d"). InnerVolumeSpecName "kube-api-access-m29h9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:25.960299 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.960269    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9"
Oct 30 19:30:25.989649 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:25.989528    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt"
Oct 30 19:30:26.022510 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.022477    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/652b5102-d21a-4673-b955-fb005179323e-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/652b5102-d21a-4673-b955-fb005179323e-maas-api-service-ca\") pod \"652b5102-d21a-4673-b955-fb005179323e\" (UID: \"652b5102-d21a-4673-b955-fb005179323e\") "
Oct 30 19:30:26.022690 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.022620    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/652b5102-d21a-4673-b955-fb005179323e-kube-api-access-9w4sv\" (UniqueName: \"kubernetes.io/projected/652b5102-d21a-4673-b955-fb005179323e-kube-api-access-9w4sv\") pod \"652b5102-d21a-4673-b955-fb005179323e\" (UID: \"652b5102-d21a-4673-b955-fb005179323e\") "
Oct 30 19:30:26.022882 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.022858    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/652b5102-d21a-4673-b955-fb005179323e-maas-api-service-ca" pod "652b5102-d21a-4673-b955-fb005179323e" (UID: "652b5102-d21a-4673-b955-fb005179323e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:26.023343 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.023319    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d-maas-api-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:26.023458 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.023345    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/652b5102-d21a-4673-b955-fb005179323e-maas-api-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:26.023458 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.023358    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m29h9\" (UniqueName: \"kubernetes.io/projected/3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d-kube-api-access-m29h9\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:26.025001 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.024979    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/652b5102-d21a-4673-b955-fb005179323e-kube-api-access-9w4sv" pod "652b5102-d21a-4673-b955-fb005179323e" (UID: "652b5102-d21a-4673-b955-fb005179323e"). InnerVolumeSpecName "kube-api-access-9w4sv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:26.124943 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.124906    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c436fb87-4351-4a27-9286-216883df85cd-kube-api-access-6mvxh\" (UniqueName: \"kubernetes.io/projected/c436fb87-4351-4a27-9286-216883df85cd-kube-api-access-6mvxh\") pod \"c436fb87-4351-4a27-9286-216883df85cd\" (UID: \"c436fb87-4351-4a27-9286-216883df85cd\") "
Oct 30 19:30:26.125118 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.124955    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c436fb87-4351-4a27-9286-216883df85cd-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c436fb87-4351-4a27-9286-216883df85cd-maas-api-service-ca\") pod \"c436fb87-4351-4a27-9286-216883df85cd\" (UID: \"c436fb87-4351-4a27-9286-216883df85cd\") "
Oct 30 19:30:26.125418 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.125393    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c436fb87-4351-4a27-9286-216883df85cd-maas-api-service-ca" pod "c436fb87-4351-4a27-9286-216883df85cd" (UID: "c436fb87-4351-4a27-9286-216883df85cd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:26.125630 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.125612    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9w4sv\" (UniqueName: \"kubernetes.io/projected/652b5102-d21a-4673-b955-fb005179323e-kube-api-access-9w4sv\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:26.125696 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.125633    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c436fb87-4351-4a27-9286-216883df85cd-maas-api-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:26.127159 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.127135    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c436fb87-4351-4a27-9286-216883df85cd-kube-api-access-6mvxh" pod "c436fb87-4351-4a27-9286-216883df85cd" (UID: "c436fb87-4351-4a27-9286-216883df85cd"). InnerVolumeSpecName "kube-api-access-6mvxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:26.226910 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.226873    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mvxh\" (UniqueName: \"kubernetes.io/projected/c436fb87-4351-4a27-9286-216883df85cd-kube-api-access-6mvxh\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:26.579310 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.579135    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn" event={"ID":"ce341e1c-1cbb-4ecc-8fda-c9088901b276","Type":"ContainerStarted","Data":"43f5c6d02b50357efcdaf094171c28dad6015367337a5e4b76757a6b5599f1db"}
Oct 30 19:30:26.611278 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.604659    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9" event={"ID":"652b5102-d21a-4673-b955-fb005179323e","Type":"ContainerDied","Data":"de2d4b380a7fe739bdc3e74b7b246873f73d73020a61e6fab505327b5944f4fa"}
Oct 30 19:30:26.611278 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.604731    4524 scope.go:122] "RemoveContainer" containerID="c3f2ca69bdfbbd7b942e6d7ea0eb842e067ef6aa7301f63128209b34a61dbbfa"
Oct 30 19:30:26.611278 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.604855    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9"
Oct 30 19:30:26.639227 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.621805    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85"]
Oct 30 19:30:26.700088 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.699880    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl" event={"ID":"27521cd5-2e0a-4836-af74-6a96ea61ba6d","Type":"ContainerStarted","Data":"211ff870d6192ba0dc0ee1d23e106c4d27e04e664cd909d39c8fec23c885ea5a"}
Oct 30 19:30:26.701941 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.701923    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf"]
Oct 30 19:30:26.706734 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.706212    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x" event={"ID":"3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d","Type":"ContainerDied","Data":"ba045246497de8df2960670c61c9a12b839b6b401daefa805ba92358bffa6765"}
Oct 30 19:30:26.706734 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.706305    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x"
Oct 30 19:30:26.710233 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.710186    4524 generic.go:356] "Generic (PLEG): container finished" podID="714dacfe-8ec6-4cbc-8aed-534f8ad30f2f" containerID="0fe67f2ba2699aecb9f08e09924e11a8b495465f3ead55cbaac30e47e40952e8" exitCode=60
Oct 30 19:30:26.711689 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.710422    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf" event={"ID":"714dacfe-8ec6-4cbc-8aed-534f8ad30f2f","Type":"ContainerDied","Data":"0fe67f2ba2699aecb9f08e09924e11a8b495465f3ead55cbaac30e47e40952e8"}
Oct 30 19:30:26.729284 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.727528    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt" event={"ID":"c436fb87-4351-4a27-9286-216883df85cd","Type":"ContainerDied","Data":"4359afc8a5ee4a29e3310084c1401c771da423019700d04b003b072ab7c61643"}
Oct 30 19:30:26.729284 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.727632    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt"
Oct 30 19:30:26.811358 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.811294    4524 scope.go:122] "RemoveContainer" containerID="e08893ad43ec0af8506bc1c2ac4964b1e778851027f1a7f3906d047e9e26cc19"
Oct 30 19:30:26.868730 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.868699    4524 scope.go:122] "RemoveContainer" containerID="895970cf4f7269eae95e75ba25085c824d679e0a05821fb01d0bff74e133bdb3"
Oct 30 19:30:26.894107 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.894079    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x"]
Oct 30 19:30:26.903215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.903182    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-8ceda0-29889810-6qw2x"]
Oct 30 19:30:26.916989 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:26.916963    4524 scope.go:122] "RemoveContainer" containerID="6b4b48b0d6a82d5d8e609f35338d7050192deefbf3a10ed2adbe065fdf154afa"
Oct 30 19:30:27.138301 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.138083    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9"]
Oct 30 19:30:27.138301 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.138233    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-33b4d7-29889810-8ttj9"]
Oct 30 19:30:27.145689 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.144491    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt"]
Oct 30 19:30:27.176194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.150813    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-afdf16-29889810-wrftt"]
Oct 30 19:30:27.462597 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.462565    4524 scope.go:122] "RemoveContainer" containerID="c1e1c7c447c9188cde79b46a9ba68042ed8cd1f37e48d2f74a2e23852d08a1c4"
Oct 30 19:30:27.464757 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.464735    4524 scope.go:122] "RemoveContainer" containerID="6497fc14a9e4f09a650abe6da015850fb53f42c5584a343d3f7452834ff0f020"
Oct 30 19:30:27.465930 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.465910    4524 scope.go:122] "RemoveContainer" containerID="4cc47c9e3eebf5bf24c109f05e145571eee752fff8c039179b66f14f0f17dacc"
Oct 30 19:30:27.652471 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.652437    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn"]
Oct 30 19:30:27.659745 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.659725    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85"
Oct 30 19:30:27.734190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.734116    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf"
Oct 30 19:30:27.748590 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.748565    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp"
Oct 30 19:30:27.799553 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.799522    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl"]
Oct 30 19:30:27.808556 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.808523    4524 generic.go:356] "Generic (PLEG): container finished" podID="ce341e1c-1cbb-4ecc-8fda-c9088901b276" containerID="43f5c6d02b50357efcdaf094171c28dad6015367337a5e4b76757a6b5599f1db" exitCode=60
Oct 30 19:30:27.865368 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.827220    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn" event={"ID":"ce341e1c-1cbb-4ecc-8fda-c9088901b276","Type":"ContainerDied","Data":"43f5c6d02b50357efcdaf094171c28dad6015367337a5e4b76757a6b5599f1db"}
Oct 30 19:30:27.865497 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.865474    4524 scope.go:122] "RemoveContainer" containerID="a6d3c3c02103ea9a63d273f6d47b5805a0e0763c4cdc3afda6f9366a174142bf"
Oct 30 19:30:27.865622 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.827295    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f17debbb-3dd6-47a2-8799-376d5735185c-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f17debbb-3dd6-47a2-8799-376d5735185c-maas-api-service-ca\") pod \"f17debbb-3dd6-47a2-8799-376d5735185c\" (UID: \"f17debbb-3dd6-47a2-8799-376d5735185c\") "
Oct 30 19:30:27.865705 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.865677    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/714dacfe-8ec6-4cbc-8aed-534f8ad30f2f-kube-api-access-8mgf2\" (UniqueName: \"kubernetes.io/projected/714dacfe-8ec6-4cbc-8aed-534f8ad30f2f-kube-api-access-8mgf2\") pod \"714dacfe-8ec6-4cbc-8aed-534f8ad30f2f\" (UID: \"714dacfe-8ec6-4cbc-8aed-534f8ad30f2f\") "
Oct 30 19:30:27.865777 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.865755    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7543e764-a757-4962-8342-f6d9d9e0f368-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7543e764-a757-4962-8342-f6d9d9e0f368-maas-api-service-ca\") pod \"7543e764-a757-4962-8342-f6d9d9e0f368\" (UID: \"7543e764-a757-4962-8342-f6d9d9e0f368\") "
Oct 30 19:30:27.865872 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.865840    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f17debbb-3dd6-47a2-8799-376d5735185c-kube-api-access-kl5l8\" (UniqueName: \"kubernetes.io/projected/f17debbb-3dd6-47a2-8799-376d5735185c-kube-api-access-kl5l8\") pod \"f17debbb-3dd6-47a2-8799-376d5735185c\" (UID: \"f17debbb-3dd6-47a2-8799-376d5735185c\") "
Oct 30 19:30:27.870226 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.842360    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f17debbb-3dd6-47a2-8799-376d5735185c-maas-api-service-ca" pod "f17debbb-3dd6-47a2-8799-376d5735185c" (UID: "f17debbb-3dd6-47a2-8799-376d5735185c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:27.870334 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.869944    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7543e764-a757-4962-8342-f6d9d9e0f368-maas-api-service-ca" pod "7543e764-a757-4962-8342-f6d9d9e0f368" (UID: "7543e764-a757-4962-8342-f6d9d9e0f368"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:27.882257 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.878928    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f17debbb-3dd6-47a2-8799-376d5735185c-kube-api-access-kl5l8" pod "f17debbb-3dd6-47a2-8799-376d5735185c" (UID: "f17debbb-3dd6-47a2-8799-376d5735185c"). InnerVolumeSpecName "kube-api-access-kl5l8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:27.889115 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.889075    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp" event={"ID":"7543e764-a757-4962-8342-f6d9d9e0f368","Type":"ContainerDied","Data":"a259a0e9ca5f6db02faaa505b43d8da9a6267e57efea4ae083c72f5f515a815f"}
Oct 30 19:30:27.892002 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.891981    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp"
Oct 30 19:30:27.894290 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.894236    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/714dacfe-8ec6-4cbc-8aed-534f8ad30f2f-kube-api-access-8mgf2" pod "714dacfe-8ec6-4cbc-8aed-534f8ad30f2f" (UID: "714dacfe-8ec6-4cbc-8aed-534f8ad30f2f"). InnerVolumeSpecName "kube-api-access-8mgf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:27.904777 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.904752    4524 generic.go:356] "Generic (PLEG): container finished" podID="27521cd5-2e0a-4836-af74-6a96ea61ba6d" containerID="211ff870d6192ba0dc0ee1d23e106c4d27e04e664cd909d39c8fec23c885ea5a" exitCode=60
Oct 30 19:30:27.906532 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.904990    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl" event={"ID":"27521cd5-2e0a-4836-af74-6a96ea61ba6d","Type":"ContainerDied","Data":"211ff870d6192ba0dc0ee1d23e106c4d27e04e664cd909d39c8fec23c885ea5a"}
Oct 30 19:30:27.943658 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.943628    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf" event={"ID":"714dacfe-8ec6-4cbc-8aed-534f8ad30f2f","Type":"ContainerDied","Data":"c3d0cf4234344d4d008b04851adcdca8377e7847481ce8a24f37681d2994177c"}
Oct 30 19:30:27.943986 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.943968    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf"
Oct 30 19:30:27.979071 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.979039    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/714dacfe-8ec6-4cbc-8aed-534f8ad30f2f-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/714dacfe-8ec6-4cbc-8aed-534f8ad30f2f-maas-api-service-ca\") pod \"714dacfe-8ec6-4cbc-8aed-534f8ad30f2f\" (UID: \"714dacfe-8ec6-4cbc-8aed-534f8ad30f2f\") "
Oct 30 19:30:27.981485 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.981454    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7543e764-a757-4962-8342-f6d9d9e0f368-kube-api-access-dfscb\" (UniqueName: \"kubernetes.io/projected/7543e764-a757-4962-8342-f6d9d9e0f368-kube-api-access-dfscb\") pod \"7543e764-a757-4962-8342-f6d9d9e0f368\" (UID: \"7543e764-a757-4962-8342-f6d9d9e0f368\") "
Oct 30 19:30:27.982026 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.982001    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/714dacfe-8ec6-4cbc-8aed-534f8ad30f2f-maas-api-service-ca" pod "714dacfe-8ec6-4cbc-8aed-534f8ad30f2f" (UID: "714dacfe-8ec6-4cbc-8aed-534f8ad30f2f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:27.983371 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.983349    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f17debbb-3dd6-47a2-8799-376d5735185c-maas-api-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:27.983480 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.983383    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mgf2\" (UniqueName: \"kubernetes.io/projected/714dacfe-8ec6-4cbc-8aed-534f8ad30f2f-kube-api-access-8mgf2\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:27.983480 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.983402    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7543e764-a757-4962-8342-f6d9d9e0f368-maas-api-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:27.983480 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.983442    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl5l8\" (UniqueName: \"kubernetes.io/projected/f17debbb-3dd6-47a2-8799-376d5735185c-kube-api-access-kl5l8\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:27.983480 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.983458    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/714dacfe-8ec6-4cbc-8aed-534f8ad30f2f-maas-api-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:27.994056 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:27.985407    4524 scope.go:122] "RemoveContainer" containerID="18078083379d92b2ef3a586f70688e90fa9859e176adde994a8568ff6d247ee7"
Oct 30 19:30:28.137491 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.137439    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7543e764-a757-4962-8342-f6d9d9e0f368-kube-api-access-dfscb" pod "7543e764-a757-4962-8342-f6d9d9e0f368" (UID: "7543e764-a757-4962-8342-f6d9d9e0f368"). InnerVolumeSpecName "kube-api-access-dfscb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:28.255456 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.255424    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"]
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269312    4524 cpu_manager.go:414] "containerMap: removing container" podUID="652b5102-d21a-4673-b955-fb005179323e" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269340    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="652b5102-d21a-4673-b955-fb005179323e" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269354    4524 cpu_manager.go:414] "containerMap: removing container" podUID="f17debbb-3dd6-47a2-8799-376d5735185c" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269362    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f17debbb-3dd6-47a2-8799-376d5735185c" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269383    4524 cpu_manager.go:414] "containerMap: removing container" podUID="f17debbb-3dd6-47a2-8799-376d5735185c" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269390    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f17debbb-3dd6-47a2-8799-376d5735185c" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269409    4524 cpu_manager.go:414] "containerMap: removing container" podUID="714dacfe-8ec6-4cbc-8aed-534f8ad30f2f" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269416    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="714dacfe-8ec6-4cbc-8aed-534f8ad30f2f" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269436    4524 cpu_manager.go:414] "containerMap: removing container" podUID="7543e764-a757-4962-8342-f6d9d9e0f368" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269443    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7543e764-a757-4962-8342-f6d9d9e0f368" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269470    4524 cpu_manager.go:414] "containerMap: removing container" podUID="c436fb87-4351-4a27-9286-216883df85cd" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269477    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c436fb87-4351-4a27-9286-216883df85cd" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269486    4524 cpu_manager.go:414] "containerMap: removing container" podUID="714dacfe-8ec6-4cbc-8aed-534f8ad30f2f" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269493    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="714dacfe-8ec6-4cbc-8aed-534f8ad30f2f" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269504    4524 cpu_manager.go:414] "containerMap: removing container" podUID="c436fb87-4351-4a27-9286-216883df85cd" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269510    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c436fb87-4351-4a27-9286-216883df85cd" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269527    4524 scope.go:122] "RemoveContainer" containerID="fe3760c03a09db92a4c23b471230d7e8ae4a88f4b12f16b3654401709425669d"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269534    4524 cpu_manager.go:414] "containerMap: removing container" podUID="3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269543    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269572    4524 cpu_manager.go:414] "containerMap: removing container" podUID="7543e764-a757-4962-8342-f6d9d9e0f368" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269579    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7543e764-a757-4962-8342-f6d9d9e0f368" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269590    4524 cpu_manager.go:414] "containerMap: removing container" podUID="652b5102-d21a-4673-b955-fb005179323e" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269596    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="652b5102-d21a-4673-b955-fb005179323e" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269615    4524 cpu_manager.go:414] "containerMap: removing container" podUID="3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269623    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269637    4524 cpu_manager.go:414] "containerMap: removing container" podUID="714dacfe-8ec6-4cbc-8aed-534f8ad30f2f" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269644    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="714dacfe-8ec6-4cbc-8aed-534f8ad30f2f" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269666    4524 cpu_manager.go:414] "containerMap: removing container" podUID="c436fb87-4351-4a27-9286-216883df85cd" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269674    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c436fb87-4351-4a27-9286-216883df85cd" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269686    4524 cpu_manager.go:414] "containerMap: removing container" podUID="7543e764-a757-4962-8342-f6d9d9e0f368" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269692    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7543e764-a757-4962-8342-f6d9d9e0f368" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269707    4524 cpu_manager.go:414] "containerMap: removing container" podUID="3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d" containerName="cleanup"
Oct 30 19:30:28.269988 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.269714    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d" containerName="cleanup"
Oct 30 19:30:28.278396 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.275467    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d" containerName="cleanup"
Oct 30 19:30:28.278396 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.275503    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="7543e764-a757-4962-8342-f6d9d9e0f368" containerName="cleanup"
Oct 30 19:30:28.278396 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.275550    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="652b5102-d21a-4673-b955-fb005179323e" containerName="cleanup"
Oct 30 19:30:28.278396 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.275566    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="714dacfe-8ec6-4cbc-8aed-534f8ad30f2f" containerName="cleanup"
Oct 30 19:30:28.278396 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.275581    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="652b5102-d21a-4673-b955-fb005179323e" containerName="cleanup"
Oct 30 19:30:28.278396 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.275592    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="c436fb87-4351-4a27-9286-216883df85cd" containerName="cleanup"
Oct 30 19:30:28.278396 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.275601    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="7543e764-a757-4962-8342-f6d9d9e0f368" containerName="cleanup"
Oct 30 19:30:28.278396 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.275613    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d" containerName="cleanup"
Oct 30 19:30:28.278396 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.275621    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="f17debbb-3dd6-47a2-8799-376d5735185c" containerName="cleanup"
Oct 30 19:30:28.278396 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.275633    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="652b5102-d21a-4673-b955-fb005179323e" containerName="cleanup"
Oct 30 19:30:28.278396 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.275642    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="f17debbb-3dd6-47a2-8799-376d5735185c" containerName="cleanup"
Oct 30 19:30:28.278396 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.275676    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="f17debbb-3dd6-47a2-8799-376d5735185c" containerName="cleanup"
Oct 30 19:30:28.278396 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.275699    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="c436fb87-4351-4a27-9286-216883df85cd" containerName="cleanup"
Oct 30 19:30:28.278396 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.275707    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="714dacfe-8ec6-4cbc-8aed-534f8ad30f2f" containerName="cleanup"
Oct 30 19:30:28.278396 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.275716    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="c436fb87-4351-4a27-9286-216883df85cd" containerName="cleanup"
Oct 30 19:30:28.278396 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.276813    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85"
Oct 30 19:30:28.287309 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.287283    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfscb\" (UniqueName: \"kubernetes.io/projected/7543e764-a757-4962-8342-f6d9d9e0f368-kube-api-access-dfscb\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:28.305987 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.305949    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"]
Oct 30 19:30:28.306088 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.306002    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85" event={"ID":"f17debbb-3dd6-47a2-8799-376d5735185c","Type":"ContainerDied","Data":"e754acf620b409ab692ced92bfecd717e58aea8863dc350152ebf8527bb78be2"}
Oct 30 19:30:28.306165 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.306110    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"
Oct 30 19:30:28.309616 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.309599    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-02feaa-serving-cert\""
Oct 30 19:30:28.339252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.339221    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp"]
Oct 30 19:30:28.349354 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.349329    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-f61e62-29889810-4zwvp"]
Oct 30 19:30:28.675986 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.675308    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tml64\" (UniqueName: \"kubernetes.io/projected/486a1648-1812-4504-83db-1c53357bd89f-kube-api-access-tml64\") pod \"maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc\" (UID: \"486a1648-1812-4504-83db-1c53357bd89f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"
Oct 30 19:30:28.675986 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.675433    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/486a1648-1812-4504-83db-1c53357bd89f-maas-api-tls\") pod \"maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc\" (UID: \"486a1648-1812-4504-83db-1c53357bd89f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"
Oct 30 19:30:28.675986 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.675507    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/486a1648-1812-4504-83db-1c53357bd89f-metrics-certs\") pod \"maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc\" (UID: \"486a1648-1812-4504-83db-1c53357bd89f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"
Oct 30 19:30:28.675986 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.675731    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/486a1648-1812-4504-83db-1c53357bd89f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc\" (UID: \"486a1648-1812-4504-83db-1c53357bd89f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"
Oct 30 19:30:28.777440 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.777403    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/486a1648-1812-4504-83db-1c53357bd89f-maas-api-tls\") pod \"maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc\" (UID: \"486a1648-1812-4504-83db-1c53357bd89f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"
Oct 30 19:30:28.998539 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.998506    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/486a1648-1812-4504-83db-1c53357bd89f-metrics-certs\") pod \"maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc\" (UID: \"486a1648-1812-4504-83db-1c53357bd89f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"
Oct 30 19:30:28.998915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.998889    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/486a1648-1812-4504-83db-1c53357bd89f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc\" (UID: \"486a1648-1812-4504-83db-1c53357bd89f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"
Oct 30 19:30:28.999268 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:28.999245    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tml64\" (UniqueName: \"kubernetes.io/projected/486a1648-1812-4504-83db-1c53357bd89f-kube-api-access-tml64\") pod \"maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc\" (UID: \"486a1648-1812-4504-83db-1c53357bd89f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"
Oct 30 19:30:29.012195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.007380    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/486a1648-1812-4504-83db-1c53357bd89f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc\" (UID: \"486a1648-1812-4504-83db-1c53357bd89f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"
Oct 30 19:30:29.085455 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.084749    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d" path="/var/lib/kubelet/pods/3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d/volumes"
Oct 30 19:30:29.112201 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.089590    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="652b5102-d21a-4673-b955-fb005179323e" path="/var/lib/kubelet/pods/652b5102-d21a-4673-b955-fb005179323e/volumes"
Oct 30 19:30:29.133395 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.128663    4524 scope.go:122] "RemoveContainer" containerID="0fe67f2ba2699aecb9f08e09924e11a8b495465f3ead55cbaac30e47e40952e8"
Oct 30 19:30:29.140183 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.140142    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7543e764-a757-4962-8342-f6d9d9e0f368" path="/var/lib/kubelet/pods/7543e764-a757-4962-8342-f6d9d9e0f368/volumes"
Oct 30 19:30:29.141103 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.141081    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c436fb87-4351-4a27-9286-216883df85cd" path="/var/lib/kubelet/pods/c436fb87-4351-4a27-9286-216883df85cd/volumes"
Oct 30 19:30:29.141796 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.141776    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85"]
Oct 30 19:30:29.150676 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.150653    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-f4012a-29889810-cmq85"]
Oct 30 19:30:29.252647 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.251628    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf"]
Oct 30 19:30:29.333462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.333427    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/486a1648-1812-4504-83db-1c53357bd89f-metrics-certs\") pod \"maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc\" (UID: \"486a1648-1812-4504-83db-1c53357bd89f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"
Oct 30 19:30:29.336803 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.336770    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/486a1648-1812-4504-83db-1c53357bd89f-maas-api-tls\") pod \"maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc\" (UID: \"486a1648-1812-4504-83db-1c53357bd89f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"
Oct 30 19:30:29.366868 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.366828    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tml64\" (UniqueName: \"kubernetes.io/projected/486a1648-1812-4504-83db-1c53357bd89f-kube-api-access-tml64\") pod \"maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc\" (UID: \"486a1648-1812-4504-83db-1c53357bd89f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"
Oct 30 19:30:29.461398 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.454034    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl"
Oct 30 19:30:29.461398 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.456923    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn"
Oct 30 19:30:29.464366 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.464337    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-891d80-29889810-fk2pf"]
Oct 30 19:30:29.796212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.791282    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn" event={"ID":"ce341e1c-1cbb-4ecc-8fda-c9088901b276","Type":"ContainerDied","Data":"264141812a36b0e8af778d0c9ecb1da9eb8ceafbab1eb878d426be846c3156a4"}
Oct 30 19:30:29.796212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.791466    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn"
Oct 30 19:30:29.796212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.793650    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"
Oct 30 19:30:29.796212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.794690    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce341e1c-1cbb-4ecc-8fda-c9088901b276-kube-api-access-q9c8d\" (UniqueName: \"kubernetes.io/projected/ce341e1c-1cbb-4ecc-8fda-c9088901b276-kube-api-access-q9c8d\") pod \"ce341e1c-1cbb-4ecc-8fda-c9088901b276\" (UID: \"ce341e1c-1cbb-4ecc-8fda-c9088901b276\") "
Oct 30 19:30:29.799809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.798837    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce341e1c-1cbb-4ecc-8fda-c9088901b276-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce341e1c-1cbb-4ecc-8fda-c9088901b276-maas-api-service-ca\") pod \"ce341e1c-1cbb-4ecc-8fda-c9088901b276\" (UID: \"ce341e1c-1cbb-4ecc-8fda-c9088901b276\") "
Oct 30 19:30:29.799809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.798882    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/27521cd5-2e0a-4836-af74-6a96ea61ba6d-kube-api-access-4pv48\" (UniqueName: \"kubernetes.io/projected/27521cd5-2e0a-4836-af74-6a96ea61ba6d-kube-api-access-4pv48\") pod \"27521cd5-2e0a-4836-af74-6a96ea61ba6d\" (UID: \"27521cd5-2e0a-4836-af74-6a96ea61ba6d\") "
Oct 30 19:30:29.799809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.798962    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/27521cd5-2e0a-4836-af74-6a96ea61ba6d-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27521cd5-2e0a-4836-af74-6a96ea61ba6d-maas-api-service-ca\") pod \"27521cd5-2e0a-4836-af74-6a96ea61ba6d\" (UID: \"27521cd5-2e0a-4836-af74-6a96ea61ba6d\") "
Oct 30 19:30:29.818519 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.800444    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce341e1c-1cbb-4ecc-8fda-c9088901b276-maas-api-service-ca" pod "ce341e1c-1cbb-4ecc-8fda-c9088901b276" (UID: "ce341e1c-1cbb-4ecc-8fda-c9088901b276"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:29.865023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.864956    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27521cd5-2e0a-4836-af74-6a96ea61ba6d-maas-api-service-ca" pod "27521cd5-2e0a-4836-af74-6a96ea61ba6d" (UID: "27521cd5-2e0a-4836-af74-6a96ea61ba6d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:29.910474 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.909867    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce341e1c-1cbb-4ecc-8fda-c9088901b276-maas-api-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:29.910474 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.909903    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/27521cd5-2e0a-4836-af74-6a96ea61ba6d-maas-api-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:29.920819 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.920790    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl" event={"ID":"27521cd5-2e0a-4836-af74-6a96ea61ba6d","Type":"ContainerDied","Data":"56eea5c104f050aed74c3803f25c6761b14731d0d0d39e04b45abee12a9a5402"}
Oct 30 19:30:29.921206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.921155    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl"
Oct 30 19:30:29.927087 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:29.927065    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp" event={"ID":"2d5c4eb6-06df-4544-84de-78b3c8b0f4e9","Type":"ContainerStarted","Data":"5d48ae844dd74c7bdf9c1161635d61ced0c2333c775868d75c8ecf2a4a596dce"}
Oct 30 19:30:30.039527 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:30.039431    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce341e1c-1cbb-4ecc-8fda-c9088901b276-kube-api-access-q9c8d" pod "ce341e1c-1cbb-4ecc-8fda-c9088901b276" (UID: "ce341e1c-1cbb-4ecc-8fda-c9088901b276"). InnerVolumeSpecName "kube-api-access-q9c8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:30.151525 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:30.148258    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9c8d\" (UniqueName: \"kubernetes.io/projected/ce341e1c-1cbb-4ecc-8fda-c9088901b276-kube-api-access-q9c8d\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:30.151525 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:30.150413    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27521cd5-2e0a-4836-af74-6a96ea61ba6d-kube-api-access-4pv48" pod "27521cd5-2e0a-4836-af74-6a96ea61ba6d" (UID: "27521cd5-2e0a-4836-af74-6a96ea61ba6d"). InnerVolumeSpecName "kube-api-access-4pv48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:30.488218 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:30.480765    4524 scope.go:122] "RemoveContainer" containerID="dfefe5c860b0ee86a753273e0564e68153971ed1f7117345e4b9b6445db0d540"
Oct 30 19:30:30.527146 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:30.527074    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pv48\" (UniqueName: \"kubernetes.io/projected/27521cd5-2e0a-4836-af74-6a96ea61ba6d-kube-api-access-4pv48\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:30.603231 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:30.602867    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="714dacfe-8ec6-4cbc-8aed-534f8ad30f2f" path="/var/lib/kubelet/pods/714dacfe-8ec6-4cbc-8aed-534f8ad30f2f/volumes"
Oct 30 19:30:30.603994 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:30.603970    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f17debbb-3dd6-47a2-8799-376d5735185c" path="/var/lib/kubelet/pods/f17debbb-3dd6-47a2-8799-376d5735185c/volumes"
Oct 30 19:30:30.770785 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:30.770760    4524 scope.go:122] "RemoveContainer" containerID="43f5c6d02b50357efcdaf094171c28dad6015367337a5e4b76757a6b5599f1db"
Oct 30 19:30:30.785105 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:30.784498    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"
Oct 30 19:30:30.813374 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:30.813049    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl"]
Oct 30 19:30:30.946603 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:30.946539    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889810-hl4xl"]
Oct 30 19:30:30.955816 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:30.955753    4524 scope.go:122] "RemoveContainer" containerID="211ff870d6192ba0dc0ee1d23e106c4d27e04e664cd909d39c8fec23c885ea5a"
Oct 30 19:30:31.263886 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.263099    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp"]
Oct 30 19:30:31.288376 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.288350    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn"]
Oct 30 19:30:31.302304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.296632    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w0-2405fb-29889810-sspdn"]
Oct 30 19:30:31.302304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.301429    4524 generic.go:356] "Generic (PLEG): container finished" podID="60537a63-2e4e-4441-a28e-2389fe0d0e93" containerID="f0d2b621aabb579bf9baba52ddf4254bed1f53137d0eb36280094e5f97c68924" exitCode=60
Oct 30 19:30:31.302304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.301531    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw" event={"ID":"60537a63-2e4e-4441-a28e-2389fe0d0e93","Type":"ContainerDied","Data":"f0d2b621aabb579bf9baba52ddf4254bed1f53137d0eb36280094e5f97c68924"}
Oct 30 19:30:31.302304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.301598    4524 scope.go:122] "RemoveContainer" containerID="c1e1c7c447c9188cde79b46a9ba68042ed8cd1f37e48d2f74a2e23852d08a1c4"
Oct 30 19:30:31.305546 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.305035    4524 scope.go:122] "RemoveContainer" containerID="f0d2b621aabb579bf9baba52ddf4254bed1f53137d0eb36280094e5f97c68924"
Oct 30 19:30:31.306774 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:31.306744    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw_odh-ai-gateway-infra(60537a63-2e4e-4441-a28e-2389fe0d0e93)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw" podUID="60537a63-2e4e-4441-a28e-2389fe0d0e93"
Oct 30 19:30:31.314270 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.314243    4524 generic.go:356] "Generic (PLEG): container finished" podID="2d5c4eb6-06df-4544-84de-78b3c8b0f4e9" containerID="5d48ae844dd74c7bdf9c1161635d61ced0c2333c775868d75c8ecf2a4a596dce" exitCode=60
Oct 30 19:30:31.315146 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.315123    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp" event={"ID":"2d5c4eb6-06df-4544-84de-78b3c8b0f4e9","Type":"ContainerDied","Data":"5d48ae844dd74c7bdf9c1161635d61ced0c2333c775868d75c8ecf2a4a596dce"}
Oct 30 19:30:31.330844 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.330815    4524 generic.go:356] "Generic (PLEG): container finished" podID="a22cffda-4a00-4d56-a975-6ad1ab789847" containerID="ba615c6c3c03a65fcf38021ef84395969132afb9911816440002edac5d69ad74" exitCode=60
Oct 30 19:30:31.331058 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.331041    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc" event={"ID":"a22cffda-4a00-4d56-a975-6ad1ab789847","Type":"ContainerDied","Data":"ba615c6c3c03a65fcf38021ef84395969132afb9911816440002edac5d69ad74"}
Oct 30 19:30:31.331748 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.331732    4524 scope.go:122] "RemoveContainer" containerID="ba615c6c3c03a65fcf38021ef84395969132afb9911816440002edac5d69ad74"
Oct 30 19:30:31.332492 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:31.332468    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc_odh-ai-gateway-infra(a22cffda-4a00-4d56-a975-6ad1ab789847)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc" podUID="a22cffda-4a00-4d56-a975-6ad1ab789847"
Oct 30 19:30:31.459008 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.458980    4524 scope.go:122] "RemoveContainer" containerID="4cc47c9e3eebf5bf24c109f05e145571eee752fff8c039179b66f14f0f17dacc"
Oct 30 19:30:31.497069 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.497040    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"]
Oct 30 19:30:31.517813 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.507578    4524 scope.go:122] "RemoveContainer" containerID="6497fc14a9e4f09a650abe6da015850fb53f42c5584a343d3f7452834ff0f020"
Oct 30 19:30:31.649901 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.649876    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp"
Oct 30 19:30:31.762220 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.762129    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d5c4eb6-06df-4544-84de-78b3c8b0f4e9-kube-api-access-lsnvj\" (UniqueName: \"kubernetes.io/projected/2d5c4eb6-06df-4544-84de-78b3c8b0f4e9-kube-api-access-lsnvj\") pod \"2d5c4eb6-06df-4544-84de-78b3c8b0f4e9\" (UID: \"2d5c4eb6-06df-4544-84de-78b3c8b0f4e9\") "
Oct 30 19:30:31.762364 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.762324    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2d5c4eb6-06df-4544-84de-78b3c8b0f4e9-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d5c4eb6-06df-4544-84de-78b3c8b0f4e9-maas-api-service-ca\") pod \"2d5c4eb6-06df-4544-84de-78b3c8b0f4e9\" (UID: \"2d5c4eb6-06df-4544-84de-78b3c8b0f4e9\") "
Oct 30 19:30:31.762685 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.762650    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d5c4eb6-06df-4544-84de-78b3c8b0f4e9-maas-api-service-ca" pod "2d5c4eb6-06df-4544-84de-78b3c8b0f4e9" (UID: "2d5c4eb6-06df-4544-84de-78b3c8b0f4e9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:31.762948 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.762931    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d5c4eb6-06df-4544-84de-78b3c8b0f4e9-maas-api-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:31.764300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.764272    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d5c4eb6-06df-4544-84de-78b3c8b0f4e9-kube-api-access-lsnvj" pod "2d5c4eb6-06df-4544-84de-78b3c8b0f4e9" (UID: "2d5c4eb6-06df-4544-84de-78b3c8b0f4e9"). InnerVolumeSpecName "kube-api-access-lsnvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:31.864041 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:31.864005    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsnvj\" (UniqueName: \"kubernetes.io/projected/2d5c4eb6-06df-4544-84de-78b3c8b0f4e9-kube-api-access-lsnvj\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:32.344788 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.344757    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp"
Oct 30 19:30:32.344934 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.344765    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp" event={"ID":"2d5c4eb6-06df-4544-84de-78b3c8b0f4e9","Type":"ContainerDied","Data":"36edfbb142e91d3533b4d590f2b31c7b9fe33cd85fc539e894cde3e20a473edc"}
Oct 30 19:30:32.344934 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.344887    4524 scope.go:122] "RemoveContainer" containerID="5d48ae844dd74c7bdf9c1161635d61ced0c2333c775868d75c8ecf2a4a596dce"
Oct 30 19:30:32.346547 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.346519    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc" event={"ID":"486a1648-1812-4504-83db-1c53357bd89f","Type":"ContainerStarted","Data":"3c69465f581cfd7e45f7ebe7b50bb484d082396076eb169070f352a97682f7c5"}
Oct 30 19:30:32.346674 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.346571    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc" event={"ID":"486a1648-1812-4504-83db-1c53357bd89f","Type":"ContainerStarted","Data":"f51e89d09e34001e86a0eac4d8fe75dec453ce19dee01e45c9b0c20c1325ed08"}
Oct 30 19:30:32.346674 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.346649    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"
Oct 30 19:30:32.363645 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.363621    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw"]
Oct 30 19:30:32.370026 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.369971    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc" podStartSLOduration=4.76178554 podStartE2EDuration="5.369952835s" totalImagesPullingTime="608.167295ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:32.367120479 +0000 UTC m=+1464.108111560" watchObservedRunningTime="2026-10-30 19:30:32.369952835 +0000 UTC m=+1464.110943915"
Oct 30 19:30:32.399165 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.399133    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="27521cd5-2e0a-4836-af74-6a96ea61ba6d" path="/var/lib/kubelet/pods/27521cd5-2e0a-4836-af74-6a96ea61ba6d/volumes"
Oct 30 19:30:32.400071 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.400049    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce341e1c-1cbb-4ecc-8fda-c9088901b276" path="/var/lib/kubelet/pods/ce341e1c-1cbb-4ecc-8fda-c9088901b276/volumes"
Oct 30 19:30:32.417434 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.417327    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp"]
Oct 30 19:30:32.425002 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.424441    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-e3586f-29889810-t6hmp"]
Oct 30 19:30:32.470437 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.470403    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc"]
Oct 30 19:30:32.714326 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.714295    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw"
Oct 30 19:30:32.796036 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.794778    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60537a63-2e4e-4441-a28e-2389fe0d0e93-kube-api-access-ggtxs\" (UniqueName: \"kubernetes.io/projected/60537a63-2e4e-4441-a28e-2389fe0d0e93-kube-api-access-ggtxs\") pod \"60537a63-2e4e-4441-a28e-2389fe0d0e93\" (UID: \"60537a63-2e4e-4441-a28e-2389fe0d0e93\") "
Oct 30 19:30:32.796036 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.795027    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60537a63-2e4e-4441-a28e-2389fe0d0e93-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60537a63-2e4e-4441-a28e-2389fe0d0e93-maas-api-service-ca\") pod \"60537a63-2e4e-4441-a28e-2389fe0d0e93\" (UID: \"60537a63-2e4e-4441-a28e-2389fe0d0e93\") "
Oct 30 19:30:32.796495 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.796452    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60537a63-2e4e-4441-a28e-2389fe0d0e93-maas-api-service-ca" pod "60537a63-2e4e-4441-a28e-2389fe0d0e93" (UID: "60537a63-2e4e-4441-a28e-2389fe0d0e93"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:32.798415 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.798379    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60537a63-2e4e-4441-a28e-2389fe0d0e93-kube-api-access-ggtxs" pod "60537a63-2e4e-4441-a28e-2389fe0d0e93" (UID: "60537a63-2e4e-4441-a28e-2389fe0d0e93"). InnerVolumeSpecName "kube-api-access-ggtxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:32.840457 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.840436    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc"
Oct 30 19:30:32.896788 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.896744    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a22cffda-4a00-4d56-a975-6ad1ab789847-kube-api-access-6wk2b\" (UniqueName: \"kubernetes.io/projected/a22cffda-4a00-4d56-a975-6ad1ab789847-kube-api-access-6wk2b\") pod \"a22cffda-4a00-4d56-a975-6ad1ab789847\" (UID: \"a22cffda-4a00-4d56-a975-6ad1ab789847\") "
Oct 30 19:30:32.897309 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.896962    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a22cffda-4a00-4d56-a975-6ad1ab789847-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a22cffda-4a00-4d56-a975-6ad1ab789847-maas-api-service-ca\") pod \"a22cffda-4a00-4d56-a975-6ad1ab789847\" (UID: \"a22cffda-4a00-4d56-a975-6ad1ab789847\") "
Oct 30 19:30:32.897566 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.897536    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggtxs\" (UniqueName: \"kubernetes.io/projected/60537a63-2e4e-4441-a28e-2389fe0d0e93-kube-api-access-ggtxs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:32.897566 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.897560    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/60537a63-2e4e-4441-a28e-2389fe0d0e93-maas-api-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:32.897849 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.897831    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a22cffda-4a00-4d56-a975-6ad1ab789847-maas-api-service-ca" pod "a22cffda-4a00-4d56-a975-6ad1ab789847" (UID: "a22cffda-4a00-4d56-a975-6ad1ab789847"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:32.899959 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.899940    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a22cffda-4a00-4d56-a975-6ad1ab789847-kube-api-access-6wk2b" pod "a22cffda-4a00-4d56-a975-6ad1ab789847" (UID: "a22cffda-4a00-4d56-a975-6ad1ab789847"). InnerVolumeSpecName "kube-api-access-6wk2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:32.999317 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.999232    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a22cffda-4a00-4d56-a975-6ad1ab789847-maas-api-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:32.999317 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:32.999262    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wk2b\" (UniqueName: \"kubernetes.io/projected/a22cffda-4a00-4d56-a975-6ad1ab789847-kube-api-access-6wk2b\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:33.357921 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:33.357886    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw"
Oct 30 19:30:33.357921 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:33.357912    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw" event={"ID":"60537a63-2e4e-4441-a28e-2389fe0d0e93","Type":"ContainerDied","Data":"d9c4c08f86206c59c55e667768a8ac1708b48561ac57a5f67d0e2e1d93f4a756"}
Oct 30 19:30:33.358140 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:33.357995    4524 scope.go:122] "RemoveContainer" containerID="f0d2b621aabb579bf9baba52ddf4254bed1f53137d0eb36280094e5f97c68924"
Oct 30 19:30:33.361082 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:33.361061    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc"
Oct 30 19:30:33.361251 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:33.361100    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc" event={"ID":"a22cffda-4a00-4d56-a975-6ad1ab789847","Type":"ContainerDied","Data":"bb9d91a7e0d08544c5362b003813740e717898d501b7255c6aa480ae6b341227"}
Oct 30 19:30:33.372704 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:33.372681    4524 scope.go:122] "RemoveContainer" containerID="ba615c6c3c03a65fcf38021ef84395969132afb9911816440002edac5d69ad74"
Oct 30 19:30:33.389711 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:33.389685    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw"]
Oct 30 19:30:33.407289 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:33.407263    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8e6159-29889810-x5qqw"]
Oct 30 19:30:33.422843 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:33.422821    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc"]
Oct 30 19:30:33.428309 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:33.428289    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-8f3ff4-29889810-twpsc"]
Oct 30 19:30:34.394022 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:34.393987    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2d5c4eb6-06df-4544-84de-78b3c8b0f4e9" path="/var/lib/kubelet/pods/2d5c4eb6-06df-4544-84de-78b3c8b0f4e9/volumes"
Oct 30 19:30:34.394473 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:34.394447    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60537a63-2e4e-4441-a28e-2389fe0d0e93" path="/var/lib/kubelet/pods/60537a63-2e4e-4441-a28e-2389fe0d0e93/volumes"
Oct 30 19:30:34.394901 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:34.394884    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a22cffda-4a00-4d56-a975-6ad1ab789847" path="/var/lib/kubelet/pods/a22cffda-4a00-4d56-a975-6ad1ab789847/volumes"
Oct 30 19:30:38.366969 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:38.366939    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"
Oct 30 19:30:41.119937 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:41.118846    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"]
Oct 30 19:30:41.119937 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:41.119106    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv" podUID="08ffae59-b193-4e72-a5a2-ed99d33366f1" containerName="maas-api" containerID="cri-o://ba655ea2981c26899f87928104654c1df27dfea810638928939a5308cf12a7dd" gracePeriod=30
Oct 30 19:30:41.917472 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:41.917435    4524 generic.go:356] "Generic (PLEG): container finished" podID="08ffae59-b193-4e72-a5a2-ed99d33366f1" containerID="ba655ea2981c26899f87928104654c1df27dfea810638928939a5308cf12a7dd" exitCode=0
Oct 30 19:30:41.917916 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:41.917890    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv" event={"ID":"08ffae59-b193-4e72-a5a2-ed99d33366f1","Type":"ContainerDied","Data":"ba655ea2981c26899f87928104654c1df27dfea810638928939a5308cf12a7dd"}
Oct 30 19:30:42.079069 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.079000    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"
Oct 30 19:30:42.104013 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.103980    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"]
Oct 30 19:30:42.106985 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.106953    4524 cpu_manager.go:414] "containerMap: removing container" podUID="652b5102-d21a-4673-b955-fb005179323e" containerName="cleanup"
Oct 30 19:30:42.106985 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.106981    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="652b5102-d21a-4673-b955-fb005179323e" containerName="cleanup"
Oct 30 19:30:42.107300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107002    4524 cpu_manager.go:414] "containerMap: removing container" podUID="f17debbb-3dd6-47a2-8799-376d5735185c" containerName="cleanup"
Oct 30 19:30:42.107300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107020    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f17debbb-3dd6-47a2-8799-376d5735185c" containerName="cleanup"
Oct 30 19:30:42.107300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107041    4524 cpu_manager.go:414] "containerMap: removing container" podUID="a22cffda-4a00-4d56-a975-6ad1ab789847" containerName="cleanup"
Oct 30 19:30:42.107300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107048    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a22cffda-4a00-4d56-a975-6ad1ab789847" containerName="cleanup"
Oct 30 19:30:42.107300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107063    4524 cpu_manager.go:414] "containerMap: removing container" podUID="2d5c4eb6-06df-4544-84de-78b3c8b0f4e9" containerName="cleanup"
Oct 30 19:30:42.107300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107070    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d5c4eb6-06df-4544-84de-78b3c8b0f4e9" containerName="cleanup"
Oct 30 19:30:42.107300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107081    4524 cpu_manager.go:414] "containerMap: removing container" podUID="ce341e1c-1cbb-4ecc-8fda-c9088901b276" containerName="cleanup"
Oct 30 19:30:42.107300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107087    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce341e1c-1cbb-4ecc-8fda-c9088901b276" containerName="cleanup"
Oct 30 19:30:42.107300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107103    4524 cpu_manager.go:414] "containerMap: removing container" podUID="27521cd5-2e0a-4836-af74-6a96ea61ba6d" containerName="cleanup"
Oct 30 19:30:42.107300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107109    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="27521cd5-2e0a-4836-af74-6a96ea61ba6d" containerName="cleanup"
Oct 30 19:30:42.107300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107123    4524 cpu_manager.go:414] "containerMap: removing container" podUID="60537a63-2e4e-4441-a28e-2389fe0d0e93" containerName="cleanup"
Oct 30 19:30:42.107300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107130    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60537a63-2e4e-4441-a28e-2389fe0d0e93" containerName="cleanup"
Oct 30 19:30:42.107300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107145    4524 cpu_manager.go:414] "containerMap: removing container" podUID="a22cffda-4a00-4d56-a975-6ad1ab789847" containerName="cleanup"
Oct 30 19:30:42.107300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107151    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a22cffda-4a00-4d56-a975-6ad1ab789847" containerName="cleanup"
Oct 30 19:30:42.107300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107166    4524 cpu_manager.go:414] "containerMap: removing container" podUID="60537a63-2e4e-4441-a28e-2389fe0d0e93" containerName="cleanup"
Oct 30 19:30:42.107300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107189    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60537a63-2e4e-4441-a28e-2389fe0d0e93" containerName="cleanup"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107335    4524 cpu_manager.go:414] "containerMap: removing container" podUID="08ffae59-b193-4e72-a5a2-ed99d33366f1" containerName="maas-api"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107344    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08ffae59-b193-4e72-a5a2-ed99d33366f1" containerName="maas-api"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107364    4524 cpu_manager.go:414] "containerMap: removing container" podUID="2d5c4eb6-06df-4544-84de-78b3c8b0f4e9" containerName="cleanup"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107370    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d5c4eb6-06df-4544-84de-78b3c8b0f4e9" containerName="cleanup"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107401    4524 cpu_manager.go:414] "containerMap: removing container" podUID="ce341e1c-1cbb-4ecc-8fda-c9088901b276" containerName="cleanup"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107408    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce341e1c-1cbb-4ecc-8fda-c9088901b276" containerName="cleanup"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107422    4524 cpu_manager.go:414] "containerMap: removing container" podUID="27521cd5-2e0a-4836-af74-6a96ea61ba6d" containerName="cleanup"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107429    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="27521cd5-2e0a-4836-af74-6a96ea61ba6d" containerName="cleanup"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107604    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="a22cffda-4a00-4d56-a975-6ad1ab789847" containerName="cleanup"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107616    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d5c4eb6-06df-4544-84de-78b3c8b0f4e9" containerName="cleanup"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107626    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce341e1c-1cbb-4ecc-8fda-c9088901b276" containerName="cleanup"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107636    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="27521cd5-2e0a-4836-af74-6a96ea61ba6d" containerName="cleanup"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107644    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="60537a63-2e4e-4441-a28e-2389fe0d0e93" containerName="cleanup"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107651    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="a22cffda-4a00-4d56-a975-6ad1ab789847" containerName="cleanup"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107659    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="714dacfe-8ec6-4cbc-8aed-534f8ad30f2f" containerName="cleanup"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107667    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce341e1c-1cbb-4ecc-8fda-c9088901b276" containerName="cleanup"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107675    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="60537a63-2e4e-4441-a28e-2389fe0d0e93" containerName="cleanup"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107682    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="08ffae59-b193-4e72-a5a2-ed99d33366f1" containerName="maas-api"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107696    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d5c4eb6-06df-4544-84de-78b3c8b0f4e9" containerName="cleanup"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107704    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d5c4eb6-06df-4544-84de-78b3c8b0f4e9" containerName="cleanup"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107713    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="27521cd5-2e0a-4836-af74-6a96ea61ba6d" containerName="cleanup"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107724    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="7543e764-a757-4962-8342-f6d9d9e0f368" containerName="cleanup"
Oct 30 19:30:42.108205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.107733    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="3aebf7f1-3afe-4a1b-97c2-df22c5f8ec0d" containerName="cleanup"
Oct 30 19:30:42.140195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.138809    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"
Oct 30 19:30:42.140195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.139854    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/08ffae59-b193-4e72-a5a2-ed99d33366f1-kube-api-access-c2gvl\" (UniqueName: \"kubernetes.io/projected/08ffae59-b193-4e72-a5a2-ed99d33366f1-kube-api-access-c2gvl\") pod \"08ffae59-b193-4e72-a5a2-ed99d33366f1\" (UID: \"08ffae59-b193-4e72-a5a2-ed99d33366f1\") "
Oct 30 19:30:42.140195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.139920    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/08ffae59-b193-4e72-a5a2-ed99d33366f1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/08ffae59-b193-4e72-a5a2-ed99d33366f1-metrics-certs\") pod \"08ffae59-b193-4e72-a5a2-ed99d33366f1\" (UID: \"08ffae59-b193-4e72-a5a2-ed99d33366f1\") "
Oct 30 19:30:42.140195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.140112    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/08ffae59-b193-4e72-a5a2-ed99d33366f1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08ffae59-b193-4e72-a5a2-ed99d33366f1-maas-api-tls\") pod \"08ffae59-b193-4e72-a5a2-ed99d33366f1\" (UID: \"08ffae59-b193-4e72-a5a2-ed99d33366f1\") "
Oct 30 19:30:42.140195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.140192    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/08ffae59-b193-4e72-a5a2-ed99d33366f1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08ffae59-b193-4e72-a5a2-ed99d33366f1-openshift-service-ca\") pod \"08ffae59-b193-4e72-a5a2-ed99d33366f1\" (UID: \"08ffae59-b193-4e72-a5a2-ed99d33366f1\") "
Oct 30 19:30:42.142688 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.142655    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08ffae59-b193-4e72-a5a2-ed99d33366f1-openshift-service-ca" pod "08ffae59-b193-4e72-a5a2-ed99d33366f1" (UID: "08ffae59-b193-4e72-a5a2-ed99d33366f1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:42.143016 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.142994    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-d52364-serving-cert\""
Oct 30 19:30:42.143298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.143281    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"]
Oct 30 19:30:42.176060 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.175718    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08ffae59-b193-4e72-a5a2-ed99d33366f1-metrics-certs" pod "08ffae59-b193-4e72-a5a2-ed99d33366f1" (UID: "08ffae59-b193-4e72-a5a2-ed99d33366f1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:30:42.176276 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.176057    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08ffae59-b193-4e72-a5a2-ed99d33366f1-kube-api-access-c2gvl" pod "08ffae59-b193-4e72-a5a2-ed99d33366f1" (UID: "08ffae59-b193-4e72-a5a2-ed99d33366f1"). InnerVolumeSpecName "kube-api-access-c2gvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:42.195369 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.194279    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08ffae59-b193-4e72-a5a2-ed99d33366f1-maas-api-tls" pod "08ffae59-b193-4e72-a5a2-ed99d33366f1" (UID: "08ffae59-b193-4e72-a5a2-ed99d33366f1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:30:42.243399 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.243373    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b536022f-7c4e-436f-a804-1c7bf324e0f0-metrics-certs\") pod \"maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg\" (UID: \"b536022f-7c4e-436f-a804-1c7bf324e0f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"
Oct 30 19:30:42.243523 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.243486    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2cpr\" (UniqueName: \"kubernetes.io/projected/b536022f-7c4e-436f-a804-1c7bf324e0f0-kube-api-access-z2cpr\") pod \"maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg\" (UID: \"b536022f-7c4e-436f-a804-1c7bf324e0f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"
Oct 30 19:30:42.243600 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.243587    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b536022f-7c4e-436f-a804-1c7bf324e0f0-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg\" (UID: \"b536022f-7c4e-436f-a804-1c7bf324e0f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"
Oct 30 19:30:42.243816 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.243796    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b536022f-7c4e-436f-a804-1c7bf324e0f0-maas-api-tls\") pod \"maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg\" (UID: \"b536022f-7c4e-436f-a804-1c7bf324e0f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"
Oct 30 19:30:42.243975 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.243960    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2gvl\" (UniqueName: \"kubernetes.io/projected/08ffae59-b193-4e72-a5a2-ed99d33366f1-kube-api-access-c2gvl\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:42.244040 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.243981    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08ffae59-b193-4e72-a5a2-ed99d33366f1-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:42.244040 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.243997    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08ffae59-b193-4e72-a5a2-ed99d33366f1-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:42.244040 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.244013    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08ffae59-b193-4e72-a5a2-ed99d33366f1-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:42.345456 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.345416    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b536022f-7c4e-436f-a804-1c7bf324e0f0-maas-api-tls\") pod \"maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg\" (UID: \"b536022f-7c4e-436f-a804-1c7bf324e0f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"
Oct 30 19:30:42.345649 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.345489    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b536022f-7c4e-436f-a804-1c7bf324e0f0-metrics-certs\") pod \"maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg\" (UID: \"b536022f-7c4e-436f-a804-1c7bf324e0f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"
Oct 30 19:30:42.345649 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.345571    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2cpr\" (UniqueName: \"kubernetes.io/projected/b536022f-7c4e-436f-a804-1c7bf324e0f0-kube-api-access-z2cpr\") pod \"maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg\" (UID: \"b536022f-7c4e-436f-a804-1c7bf324e0f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"
Oct 30 19:30:42.345763 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.345691    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b536022f-7c4e-436f-a804-1c7bf324e0f0-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg\" (UID: \"b536022f-7c4e-436f-a804-1c7bf324e0f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"
Oct 30 19:30:42.347145 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.346598    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b536022f-7c4e-436f-a804-1c7bf324e0f0-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg\" (UID: \"b536022f-7c4e-436f-a804-1c7bf324e0f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"
Oct 30 19:30:42.348775 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.348467    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b536022f-7c4e-436f-a804-1c7bf324e0f0-maas-api-tls\") pod \"maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg\" (UID: \"b536022f-7c4e-436f-a804-1c7bf324e0f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"
Oct 30 19:30:42.403206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.394953    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b536022f-7c4e-436f-a804-1c7bf324e0f0-metrics-certs\") pod \"maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg\" (UID: \"b536022f-7c4e-436f-a804-1c7bf324e0f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"
Oct 30 19:30:42.450663 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.449087    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2cpr\" (UniqueName: \"kubernetes.io/projected/b536022f-7c4e-436f-a804-1c7bf324e0f0-kube-api-access-z2cpr\") pod \"maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg\" (UID: \"b536022f-7c4e-436f-a804-1c7bf324e0f0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"
Oct 30 19:30:42.553194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.536470    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"
Oct 30 19:30:42.864103 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:30:42.864073    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb536022f_7c4e_436f_a804_1c7bf324e0f0.slice/crio-fb31475595091d8d6f698882abe260273c6e56bc7739be7976bb24cbf457641b WatchSource:0}: Error finding container fb31475595091d8d6f698882abe260273c6e56bc7739be7976bb24cbf457641b: Status 404 returned error can't find the container with id fb31475595091d8d6f698882abe260273c6e56bc7739be7976bb24cbf457641b
Oct 30 19:30:42.864946 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.864775    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"]
Oct 30 19:30:42.960766 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.960723    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg" event={"ID":"b536022f-7c4e-436f-a804-1c7bf324e0f0","Type":"ContainerStarted","Data":"fb31475595091d8d6f698882abe260273c6e56bc7739be7976bb24cbf457641b"}
Oct 30 19:30:42.962753 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.962726    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv" event={"ID":"08ffae59-b193-4e72-a5a2-ed99d33366f1","Type":"ContainerDied","Data":"e0060aeaae2893628a6ee012457e5bb1fa01990be9f5de10da5c87816538591a"}
Oct 30 19:30:42.962910 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.962783    4524 scope.go:122] "RemoveContainer" containerID="ba655ea2981c26899f87928104654c1df27dfea810638928939a5308cf12a7dd"
Oct 30 19:30:42.962910 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:42.962783    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"
Oct 30 19:30:43.011150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:43.011085    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"]
Oct 30 19:30:43.052194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:43.040364    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-8ceda0-6544f8c6cc-8vjxv"]
Oct 30 19:30:43.975618 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:43.975579    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg" event={"ID":"b536022f-7c4e-436f-a804-1c7bf324e0f0","Type":"ContainerStarted","Data":"2c608f06fc79a7a13bda907dc5f2dd3b091be1b62404c5b820e3e9a1552c55d3"}
Oct 30 19:30:43.976051 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:43.975637    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"
Oct 30 19:30:44.400464 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:44.400430    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="08ffae59-b193-4e72-a5a2-ed99d33366f1" path="/var/lib/kubelet/pods/08ffae59-b193-4e72-a5a2-ed99d33366f1/volumes"
Oct 30 19:30:45.627513 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:30:45.627472    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:30:50.143299 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:50.143266    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"
Oct 30 19:30:50.176447 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:30:50.176387    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg" podStartSLOduration=7.69323462 podStartE2EDuration="8.176373218s" totalImagesPullingTime="483.138598ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:44.000589675 +0000 UTC m=+1475.741580749" watchObservedRunningTime="2026-10-30 19:30:50.176373218 +0000 UTC m=+1481.917364296"
Oct 30 19:31:15.687075 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:31:15.687030    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:31:19.942929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:19.942883    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"]
Oct 30 19:31:19.944102 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:19.944070    4524 cpu_manager.go:414] "containerMap: removing container" podUID="a22cffda-4a00-4d56-a975-6ad1ab789847" containerName="cleanup"
Oct 30 19:31:19.944102 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:19.944092    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a22cffda-4a00-4d56-a975-6ad1ab789847" containerName="cleanup"
Oct 30 19:31:19.945793 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:19.944113    4524 cpu_manager.go:414] "containerMap: removing container" podUID="60537a63-2e4e-4441-a28e-2389fe0d0e93" containerName="cleanup"
Oct 30 19:31:19.945793 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:19.944121    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60537a63-2e4e-4441-a28e-2389fe0d0e93" containerName="cleanup"
Oct 30 19:31:19.945793 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:19.945017    4524 cpu_manager.go:414] "containerMap: removing container" podUID="2d5c4eb6-06df-4544-84de-78b3c8b0f4e9" containerName="cleanup"
Oct 30 19:31:19.945793 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:19.945054    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d5c4eb6-06df-4544-84de-78b3c8b0f4e9" containerName="cleanup"
Oct 30 19:31:19.945793 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:19.945102    4524 cpu_manager.go:414] "containerMap: removing container" podUID="ce341e1c-1cbb-4ecc-8fda-c9088901b276" containerName="cleanup"
Oct 30 19:31:19.945793 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:19.945110    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce341e1c-1cbb-4ecc-8fda-c9088901b276" containerName="cleanup"
Oct 30 19:31:19.945793 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:19.945129    4524 cpu_manager.go:414] "containerMap: removing container" podUID="27521cd5-2e0a-4836-af74-6a96ea61ba6d" containerName="cleanup"
Oct 30 19:31:19.945793 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:19.945137    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="27521cd5-2e0a-4836-af74-6a96ea61ba6d" containerName="cleanup"
Oct 30 19:31:19.945793 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:19.945401    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce341e1c-1cbb-4ecc-8fda-c9088901b276" containerName="cleanup"
Oct 30 19:31:19.945793 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:19.945417    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="a22cffda-4a00-4d56-a975-6ad1ab789847" containerName="cleanup"
Oct 30 19:31:19.945793 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:19.945433    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="60537a63-2e4e-4441-a28e-2389fe0d0e93" containerName="cleanup"
Oct 30 19:31:19.945793 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:19.945448    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="27521cd5-2e0a-4836-af74-6a96ea61ba6d" containerName="cleanup"
Oct 30 19:31:19.953422 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:19.953397    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"
Oct 30 19:31:19.960017 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:19.959966    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"]
Oct 30 19:31:19.960894 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:19.960872    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-69f3a9-serving-cert\""
Oct 30 19:31:20.057215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:20.050190    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhspx\" (UniqueName: \"kubernetes.io/projected/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-kube-api-access-mhspx\") pod \"maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv\" (UID: \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"
Oct 30 19:31:20.057215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:20.050290    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv\" (UID: \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"
Oct 30 19:31:20.057215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:20.050437    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-metrics-certs\") pod \"maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv\" (UID: \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"
Oct 30 19:31:20.057215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:20.050530    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-maas-api-tls\") pod \"maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv\" (UID: \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"
Oct 30 19:31:20.158197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:20.153630    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-maas-api-tls\") pod \"maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv\" (UID: \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"
Oct 30 19:31:20.158197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:20.153864    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhspx\" (UniqueName: \"kubernetes.io/projected/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-kube-api-access-mhspx\") pod \"maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv\" (UID: \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"
Oct 30 19:31:20.158197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:20.153933    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv\" (UID: \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"
Oct 30 19:31:20.158197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:20.154073    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-metrics-certs\") pod \"maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv\" (UID: \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"
Oct 30 19:31:20.169202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:20.164941    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-metrics-certs\") pod \"maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv\" (UID: \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"
Oct 30 19:31:20.169202 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:31:20.165080    4524 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-serving-cert: secret "maas-api-e2e-inf-b-69f3a9-serving-cert" not found
Oct 30 19:31:20.169202 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:31:20.165155    4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-maas-api-tls podName:e24b4adc-d486-4aa9-a94d-d15cc1a30efb nodeName:}" failed. No retries permitted until 2026-10-30 19:31:20.665135527 +0000 UTC m=+1512.406126598 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-maas-api-tls") pod "maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv" (UID: "e24b4adc-d486-4aa9-a94d-d15cc1a30efb") : secret "maas-api-e2e-inf-b-69f3a9-serving-cert" not found
Oct 30 19:31:20.169202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:20.166591    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv\" (UID: \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"
Oct 30 19:31:20.203853 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:20.193248    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhspx\" (UniqueName: \"kubernetes.io/projected/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-kube-api-access-mhspx\") pod \"maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv\" (UID: \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"
Oct 30 19:31:20.739715 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:20.739682    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-maas-api-tls\") pod \"maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv\" (UID: \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"
Oct 30 19:31:20.782616 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:20.777770    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-maas-api-tls\") pod \"maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv\" (UID: \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"
Oct 30 19:31:20.877306 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:20.877262    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"
Oct 30 19:31:21.174997 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:21.174967    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"]
Oct 30 19:31:21.175372 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:31:21.175237    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode24b4adc_d486_4aa9_a94d_d15cc1a30efb.slice/crio-f2cf8b5bbcdb401b8a8bcfd28fa579150d230caac6b9f7feef85c4cd1955b71b WatchSource:0}: Error finding container f2cf8b5bbcdb401b8a8bcfd28fa579150d230caac6b9f7feef85c4cd1955b71b: Status 404 returned error can't find the container with id f2cf8b5bbcdb401b8a8bcfd28fa579150d230caac6b9f7feef85c4cd1955b71b
Oct 30 19:31:21.795659 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:21.795619    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv" event={"ID":"e24b4adc-d486-4aa9-a94d-d15cc1a30efb","Type":"ContainerStarted","Data":"f2cf8b5bbcdb401b8a8bcfd28fa579150d230caac6b9f7feef85c4cd1955b71b"}
Oct 30 19:31:22.805719 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:22.805683    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv" event={"ID":"e24b4adc-d486-4aa9-a94d-d15cc1a30efb","Type":"ContainerStarted","Data":"8ebae01a749c180df287bcea399e1dc36b03e29439e7049ec86baf45ce6015bf"}
Oct 30 19:31:22.806138 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:22.805776    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"
Oct 30 19:31:22.834470 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:22.834422    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv" podStartSLOduration=3.126739156 podStartE2EDuration="3.834408664s" totalImagesPullingTime="707.669508ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:22.83074603 +0000 UTC m=+1514.571737104" watchObservedRunningTime="2026-10-30 19:31:22.834408664 +0000 UTC m=+1514.575399742"
Oct 30 19:31:26.891881 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:26.891844    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"]
Oct 30 19:31:26.892557 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:26.892501    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b" podUID="ddef5788-e8d7-4f77-9828-a5557e3cc615" containerName="maas-api" containerID="cri-o://053dcc1fc902afc430f9c5b025782a49329022caa5a686ca9443138653ad0daa" gracePeriod=30
Oct 30 19:31:27.122676 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:27.122316    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b" podUID="ddef5788-e8d7-4f77-9828-a5557e3cc615" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.123:8443/health\": dial tcp 10.133.0.123:8443: connect: connection refused"
Oct 30 19:31:27.773742 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:27.773421    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"
Oct 30 19:31:27.845599 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:27.845562    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ddef5788-e8d7-4f77-9828-a5557e3cc615-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddef5788-e8d7-4f77-9828-a5557e3cc615-metrics-certs\") pod \"ddef5788-e8d7-4f77-9828-a5557e3cc615\" (UID: \"ddef5788-e8d7-4f77-9828-a5557e3cc615\") "
Oct 30 19:31:27.845793 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:27.845684    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ddef5788-e8d7-4f77-9828-a5557e3cc615-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ddef5788-e8d7-4f77-9828-a5557e3cc615-maas-api-tls\") pod \"ddef5788-e8d7-4f77-9828-a5557e3cc615\" (UID: \"ddef5788-e8d7-4f77-9828-a5557e3cc615\") "
Oct 30 19:31:27.845858 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:27.845809    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ddef5788-e8d7-4f77-9828-a5557e3cc615-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddef5788-e8d7-4f77-9828-a5557e3cc615-openshift-service-ca\") pod \"ddef5788-e8d7-4f77-9828-a5557e3cc615\" (UID: \"ddef5788-e8d7-4f77-9828-a5557e3cc615\") "
Oct 30 19:31:27.846184 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:27.846146    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddef5788-e8d7-4f77-9828-a5557e3cc615-openshift-service-ca" pod "ddef5788-e8d7-4f77-9828-a5557e3cc615" (UID: "ddef5788-e8d7-4f77-9828-a5557e3cc615"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:31:27.846351 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:27.846333    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ddef5788-e8d7-4f77-9828-a5557e3cc615-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:27.847736 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:27.847712    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddef5788-e8d7-4f77-9828-a5557e3cc615-maas-api-tls" pod "ddef5788-e8d7-4f77-9828-a5557e3cc615" (UID: "ddef5788-e8d7-4f77-9828-a5557e3cc615"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:31:27.847836 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:27.847741    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddef5788-e8d7-4f77-9828-a5557e3cc615-metrics-certs" pod "ddef5788-e8d7-4f77-9828-a5557e3cc615" (UID: "ddef5788-e8d7-4f77-9828-a5557e3cc615"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:31:27.925058 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:27.923908    4524 generic.go:356] "Generic (PLEG): container finished" podID="ddef5788-e8d7-4f77-9828-a5557e3cc615" containerID="053dcc1fc902afc430f9c5b025782a49329022caa5a686ca9443138653ad0daa" exitCode=0
Oct 30 19:31:27.925058 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:27.924254    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b" event={"ID":"ddef5788-e8d7-4f77-9828-a5557e3cc615","Type":"ContainerDied","Data":"053dcc1fc902afc430f9c5b025782a49329022caa5a686ca9443138653ad0daa"}
Oct 30 19:31:27.925058 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:27.924316    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b" event={"ID":"ddef5788-e8d7-4f77-9828-a5557e3cc615","Type":"ContainerDied","Data":"de1d0993976aaeab3c6d4d1f18ae79340abad4c1d29304e5d1469789a21a28ce"}
Oct 30 19:31:27.925058 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:27.924356    4524 scope.go:122] "RemoveContainer" containerID="053dcc1fc902afc430f9c5b025782a49329022caa5a686ca9443138653ad0daa"
Oct 30 19:31:27.925058 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:27.924532    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"
Oct 30 19:31:27.939425 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:27.939397    4524 scope.go:122] "RemoveContainer" containerID="053dcc1fc902afc430f9c5b025782a49329022caa5a686ca9443138653ad0daa"
Oct 30 19:31:27.939738 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:31:27.939714    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"053dcc1fc902afc430f9c5b025782a49329022caa5a686ca9443138653ad0daa\": container with ID starting with 053dcc1fc902afc430f9c5b025782a49329022caa5a686ca9443138653ad0daa not found: ID does not exist" containerID="053dcc1fc902afc430f9c5b025782a49329022caa5a686ca9443138653ad0daa"
Oct 30 19:31:27.939819 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:27.939753    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"053dcc1fc902afc430f9c5b025782a49329022caa5a686ca9443138653ad0daa"} err="failed to get container status \"053dcc1fc902afc430f9c5b025782a49329022caa5a686ca9443138653ad0daa\": rpc error: code = NotFound desc = could not find container \"053dcc1fc902afc430f9c5b025782a49329022caa5a686ca9443138653ad0daa\": container with ID starting with 053dcc1fc902afc430f9c5b025782a49329022caa5a686ca9443138653ad0daa not found: ID does not exist"
Oct 30 19:31:27.947528 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:27.947495    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ddef5788-e8d7-4f77-9828-a5557e3cc615-kube-api-access-cnkt5\" (UniqueName: \"kubernetes.io/projected/ddef5788-e8d7-4f77-9828-a5557e3cc615-kube-api-access-cnkt5\") pod \"ddef5788-e8d7-4f77-9828-a5557e3cc615\" (UID: \"ddef5788-e8d7-4f77-9828-a5557e3cc615\") "
Oct 30 19:31:27.948061 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:27.948039    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ddef5788-e8d7-4f77-9828-a5557e3cc615-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:27.948144 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:27.948065    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ddef5788-e8d7-4f77-9828-a5557e3cc615-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:27.949527 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:27.949505    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddef5788-e8d7-4f77-9828-a5557e3cc615-kube-api-access-cnkt5" pod "ddef5788-e8d7-4f77-9828-a5557e3cc615" (UID: "ddef5788-e8d7-4f77-9828-a5557e3cc615"). InnerVolumeSpecName "kube-api-access-cnkt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:31:28.049242 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:28.049126    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnkt5\" (UniqueName: \"kubernetes.io/projected/ddef5788-e8d7-4f77-9828-a5557e3cc615-kube-api-access-cnkt5\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:28.249542 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:28.249500    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"]
Oct 30 19:31:28.256317 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:28.256283    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f4012a-966bbc888-pl46b"]
Oct 30 19:31:28.393645 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:28.393610    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ddef5788-e8d7-4f77-9828-a5557e3cc615" path="/var/lib/kubelet/pods/ddef5788-e8d7-4f77-9828-a5557e3cc615/volumes"
Oct 30 19:31:28.820987 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:28.820913    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"
Oct 30 19:31:30.364817 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:30.364777    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"]
Oct 30 19:31:30.366009 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:30.365987    4524 cpu_manager.go:414] "containerMap: removing container" podUID="ddef5788-e8d7-4f77-9828-a5557e3cc615" containerName="maas-api"
Oct 30 19:31:30.366143 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:30.366010    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ddef5788-e8d7-4f77-9828-a5557e3cc615" containerName="maas-api"
Oct 30 19:31:30.366143 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:30.366185    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="ddef5788-e8d7-4f77-9828-a5557e3cc615" containerName="maas-api"
Oct 30 19:31:30.394493 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:30.394460    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"
Oct 30 19:31:30.402733 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:30.402646    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg\" (UID: \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"
Oct 30 19:31:30.403061 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:30.403042    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfbkp\" (UniqueName: \"kubernetes.io/projected/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-kube-api-access-mfbkp\") pod \"maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg\" (UID: \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"
Oct 30 19:31:30.403378 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:30.403361    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-metrics-certs\") pod \"maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg\" (UID: \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"
Oct 30 19:31:30.403618 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:30.403604    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-maas-api-tls\") pod \"maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg\" (UID: \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"
Oct 30 19:31:30.411744 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:30.411720    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-c86696-serving-cert\""
Oct 30 19:31:30.416309 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:30.416289    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"]
Oct 30 19:31:30.505348 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:30.505306    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-metrics-certs\") pod \"maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg\" (UID: \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"
Oct 30 19:31:30.505566 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:30.505474    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-maas-api-tls\") pod \"maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg\" (UID: \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"
Oct 30 19:31:30.505566 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:30.505539    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg\" (UID: \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"
Oct 30 19:31:30.505702 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:31:30.505624    4524 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-serving-cert: secret "maas-api-e2e-inf-b-c86696-serving-cert" not found
Oct 30 19:31:30.505702 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:31:30.505700    4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-maas-api-tls podName:a1ae4189-16e5-4d0d-b64e-1e660176cb4f nodeName:}" failed. No retries permitted until 2026-10-30 19:31:31.005679694 +0000 UTC m=+1522.746670767 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-maas-api-tls") pod "maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg" (UID: "a1ae4189-16e5-4d0d-b64e-1e660176cb4f") : secret "maas-api-e2e-inf-b-c86696-serving-cert" not found
Oct 30 19:31:30.505822 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:30.505722    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfbkp\" (UniqueName: \"kubernetes.io/projected/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-kube-api-access-mfbkp\") pod \"maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg\" (UID: \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"
Oct 30 19:31:30.506325 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:30.506294    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg\" (UID: \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"
Oct 30 19:31:30.508404 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:30.508377    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-metrics-certs\") pod \"maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg\" (UID: \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"
Oct 30 19:31:30.520870 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:30.520849    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfbkp\" (UniqueName: \"kubernetes.io/projected/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-kube-api-access-mfbkp\") pod \"maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg\" (UID: \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"
Oct 30 19:31:31.112194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:31.110340    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-maas-api-tls\") pod \"maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg\" (UID: \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"
Oct 30 19:31:31.153193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:31.152735    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-maas-api-tls\") pod \"maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg\" (UID: \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"
Oct 30 19:31:31.332817 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:31.332779    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"
Oct 30 19:31:32.189516 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:32.182647    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"]
Oct 30 19:31:33.112091 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:33.112058    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg" event={"ID":"a1ae4189-16e5-4d0d-b64e-1e660176cb4f","Type":"ContainerStarted","Data":"d4c86e5f2cb0ba6afbab5f8deb47009532096e94579c580ac4e6683ed6376ddb"}
Oct 30 19:31:33.112308 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:33.112108    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg" event={"ID":"a1ae4189-16e5-4d0d-b64e-1e660176cb4f","Type":"ContainerStarted","Data":"6f192a41defdf02de68825bd765ab73073d73c35c1151d83d8c3640c15ad1b5f"}
Oct 30 19:31:33.112308 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:33.112295    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"
Oct 30 19:31:33.136689 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:33.136639    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg" podStartSLOduration=2.609304651 podStartE2EDuration="3.136625903s" totalImagesPullingTime="527.321252ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:33.13301713 +0000 UTC m=+1524.874008208" watchObservedRunningTime="2026-10-30 19:31:33.136625903 +0000 UTC m=+1524.877616981"
Oct 30 19:31:39.134830 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:39.134796    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"
Oct 30 19:31:45.627123 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:31:45.627082    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:31:55.621013 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.620971    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"]
Oct 30 19:31:55.627458 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.627436    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.634964 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.634944    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-ef5dca-openshift-default-dockercfg-vkm6l\""
Oct 30 19:31:55.646675 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.646650    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"]
Oct 30 19:31:55.770666 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.770624    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c3eeede-eb3c-4469-9b85-279c56d473ff-istiod-ca-cert\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.770845 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.770757    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-data\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.770845 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.770815    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-podinfo\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.770978 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.770951    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwwck\" (UniqueName: \"kubernetes.io/projected/0c3eeede-eb3c-4469-9b85-279c56d473ff-kube-api-access-xwwck\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.771053 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.771034    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-workload-socket\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.771094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.771068    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-credential-socket\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.771202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.771151    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-workload-certs\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.771302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.771270    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-token\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.771451 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.771436    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-envoy\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.872903 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.872808    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-token\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.873077 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.873009    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-envoy\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.873077 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.873061    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c3eeede-eb3c-4469-9b85-279c56d473ff-istiod-ca-cert\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.873225 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.873130    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-data\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.873225 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.873191    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-podinfo\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.873330 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.873247    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwwck\" (UniqueName: \"kubernetes.io/projected/0c3eeede-eb3c-4469-9b85-279c56d473ff-kube-api-access-xwwck\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.873330 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.873278    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-workload-socket\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.873330 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.873303    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-credential-socket\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.873477 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.873367    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-workload-certs\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.873868 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.873843    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-workload-certs\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.874770 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.874669    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c3eeede-eb3c-4469-9b85-279c56d473ff-istiod-ca-cert\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.874888 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.874786    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-data\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.875089 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.875064    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-workload-socket\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.875089 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.875080    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-credential-socket\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.876336 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.876285    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-podinfo\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.877161 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.877134    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-envoy\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.888606 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.888581    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-token\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.888841 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.888813    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwwck\" (UniqueName: \"kubernetes.io/projected/0c3eeede-eb3c-4469-9b85-279c56d473ff-kube-api-access-xwwck\") pod \"e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") " pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:55.947562 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:55.947533    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:56.293212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:56.291990    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:31:56.293212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:56.292088    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:31:56.293212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:56.292129    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:31:56.353098 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:56.348375    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"]
Oct 30 19:31:56.572281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:56.571349    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw" event={"ID":"0c3eeede-eb3c-4469-9b85-279c56d473ff","Type":"ContainerStarted","Data":"d79c4e372d503938a6c524b58c08f5ae0c9e450909bb2bcf845e6f73991b8319"}
Oct 30 19:31:56.572281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:56.571427    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw" event={"ID":"0c3eeede-eb3c-4469-9b85-279c56d473ff","Type":"ContainerStarted","Data":"004489f36637f4b0b80ae7fa824539c794588a179e70dd80229749d5d9860406"}
Oct 30 19:31:56.600639 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:56.599280    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw" podStartSLOduration=1.599260344 podStartE2EDuration="1.599260344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:31:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:31:56.592484153 +0000 UTC m=+1548.333475244" watchObservedRunningTime="2026-10-30 19:31:56.599260344 +0000 UTC m=+1548.340251425"
Oct 30 19:31:56.948184 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:56.948138    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:31:57.951201 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:57.949420    4524 patch_prober.go:29] interesting pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:31:57.951201 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:57.949514    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw" podUID="0c3eeede-eb3c-4469-9b85-279c56d473ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused"
Oct 30 19:31:58.959676 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:58.955637    4524 patch_prober.go:29] interesting pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:31:58.959676 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:58.955742    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw" podUID="0c3eeede-eb3c-4469-9b85-279c56d473ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused"
Oct 30 19:31:59.947998 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:59.947963    4524 patch_prober.go:29] interesting pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:31:59.948217 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:31:59.948015    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw" podUID="0c3eeede-eb3c-4469-9b85-279c56d473ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused"
Oct 30 19:32:00.954162 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:32:00.951778    4524 patch_prober.go:29] interesting pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:32:00.954162 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:32:00.951847    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw" podUID="0c3eeede-eb3c-4469-9b85-279c56d473ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused"
Oct 30 19:32:01.948011 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:32:01.947970    4524 patch_prober.go:29] interesting pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:32:01.948332 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:32:01.948294    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw" podUID="0c3eeede-eb3c-4469-9b85-279c56d473ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused"
Oct 30 19:32:02.951086 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:32:02.951044    4524 patch_prober.go:29] interesting pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:32:02.951706 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:32:02.951667    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw" podUID="0c3eeede-eb3c-4469-9b85-279c56d473ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused"
Oct 30 19:32:03.949922 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:32:03.949879    4524 patch_prober.go:29] interesting pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:32:03.950245 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:32:03.950207    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw" podUID="0c3eeede-eb3c-4469-9b85-279c56d473ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.145:15021/healthz/ready\": dial tcp 10.133.0.145:15021: connect: connection refused"
Oct 30 19:32:04.953237 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:32:04.953202    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:32:04.953694 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:32:04.953673    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:32:04.954461 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:32:04.954444    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:32:15.628188 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:32:15.628141    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:32:45.682827 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:32:45.682770    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:33:15.628127 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:33:15.628088    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:33:35.664193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.656512    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"]
Oct 30 19:33:35.678285 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.675285    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:35.687963 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.687611    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-76117c\"/\"openshift-service-ca.crt\""
Oct 30 19:33:35.700650 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.698756    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-76117c\"/\"kube-root-ca.crt\""
Oct 30 19:33:35.704150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.704120    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"]
Oct 30 19:33:35.708448 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.705812    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-76117c\"/\"test-model-76117c-kserve-self-signed-certs\""
Oct 30 19:33:35.708448 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.705867    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-76117c\"/\"default-dockercfg-xddq6\""
Oct 30 19:33:35.822888 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.822855    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-tmp-dir\") pod \"test-model-76117c-kserve-58cd949d8f-9tnxt\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") " pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:35.823044 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.822902    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/930da0d5-fe44-4862-ad00-2c789c18882c-tls-certs\") pod \"test-model-76117c-kserve-58cd949d8f-9tnxt\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") " pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:35.823044 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.823024    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-home\") pod \"test-model-76117c-kserve-58cd949d8f-9tnxt\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") " pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:35.823178 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.823073    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc5rg\" (UniqueName: \"kubernetes.io/projected/930da0d5-fe44-4862-ad00-2c789c18882c-kube-api-access-pc5rg\") pod \"test-model-76117c-kserve-58cd949d8f-9tnxt\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") " pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:35.823178 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.823130    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-model-cache\") pod \"test-model-76117c-kserve-58cd949d8f-9tnxt\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") " pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:35.823368 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.823350    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-dshm\") pod \"test-model-76117c-kserve-58cd949d8f-9tnxt\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") " pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:35.934204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.928382    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-dshm\") pod \"test-model-76117c-kserve-58cd949d8f-9tnxt\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") " pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:35.934204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.928468    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-tmp-dir\") pod \"test-model-76117c-kserve-58cd949d8f-9tnxt\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") " pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:35.934204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.928514    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/930da0d5-fe44-4862-ad00-2c789c18882c-tls-certs\") pod \"test-model-76117c-kserve-58cd949d8f-9tnxt\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") " pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:35.934204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.928617    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-home\") pod \"test-model-76117c-kserve-58cd949d8f-9tnxt\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") " pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:35.934204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.928679    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc5rg\" (UniqueName: \"kubernetes.io/projected/930da0d5-fe44-4862-ad00-2c789c18882c-kube-api-access-pc5rg\") pod \"test-model-76117c-kserve-58cd949d8f-9tnxt\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") " pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:35.934204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.928708    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-model-cache\") pod \"test-model-76117c-kserve-58cd949d8f-9tnxt\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") " pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:35.934204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.929221    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-model-cache\") pod \"test-model-76117c-kserve-58cd949d8f-9tnxt\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") " pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:35.934204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.932508    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-home\") pod \"test-model-76117c-kserve-58cd949d8f-9tnxt\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") " pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:35.934204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.933070    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-tmp-dir\") pod \"test-model-76117c-kserve-58cd949d8f-9tnxt\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") " pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:35.981253 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.979714    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/930da0d5-fe44-4862-ad00-2c789c18882c-tls-certs\") pod \"test-model-76117c-kserve-58cd949d8f-9tnxt\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") " pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:35.982196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.982156    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc5rg\" (UniqueName: \"kubernetes.io/projected/930da0d5-fe44-4862-ad00-2c789c18882c-kube-api-access-pc5rg\") pod \"test-model-76117c-kserve-58cd949d8f-9tnxt\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") " pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:35.994528 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:35.994452    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-dshm\") pod \"test-model-76117c-kserve-58cd949d8f-9tnxt\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") " pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:36.013776 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:36.013491    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:36.563461 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:36.563430    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"]
Oct 30 19:33:36.629039 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:36.629005    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt" event={"ID":"930da0d5-fe44-4862-ad00-2c789c18882c","Type":"ContainerStarted","Data":"437f08d58d20241c55c590634869281e390e55172b5a6ae11188834672b02ee1"}
Oct 30 19:33:37.638673 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:37.638641    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt" event={"ID":"930da0d5-fe44-4862-ad00-2c789c18882c","Type":"ContainerStarted","Data":"f1ee090517aa66d91bee1d9bfd1d06c3ab84fcc47424ac93be6ef421b4db5e36"}
Oct 30 19:33:37.672252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:37.672194    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt" podStartSLOduration=2.67216299 podStartE2EDuration="2.67216299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:33:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:33:37.66499598 +0000 UTC m=+1649.405987059" watchObservedRunningTime="2026-10-30 19:33:37.67216299 +0000 UTC m=+1649.413154072"
Oct 30 19:33:38.015023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:38.014947    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:45.700306 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:33:45.700103    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:33:46.015106 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:46.015016    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:46.029620 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:46.029592    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:33:46.798700 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:33:46.798665    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:34:15.627074 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:34:15.627038    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:34:21.233833 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:21.233772    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q"]
Oct 30 19:34:22.692940 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:22.692893    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"]
Oct 30 19:34:22.693475 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:22.693157    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh" podUID="3108bb35-3f80-4f5b-b9a4-505982b1f1ee" containerName="maas-api" containerID="cri-o://a54040016ab4f8da8c7b4919c5e742698bbe973cfa7d663e4e2f98a4a709c073" gracePeriod=30
Oct 30 19:34:23.333707 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.333680    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"
Oct 30 19:34:23.444265 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.416844    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-metrics-certs\") pod \"3108bb35-3f80-4f5b-b9a4-505982b1f1ee\" (UID: \"3108bb35-3f80-4f5b-b9a4-505982b1f1ee\") "
Oct 30 19:34:23.444265 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.416913    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-kube-api-access-sch5l\" (UniqueName: \"kubernetes.io/projected/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-kube-api-access-sch5l\") pod \"3108bb35-3f80-4f5b-b9a4-505982b1f1ee\" (UID: \"3108bb35-3f80-4f5b-b9a4-505982b1f1ee\") "
Oct 30 19:34:23.444265 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.416987    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-openshift-service-ca\") pod \"3108bb35-3f80-4f5b-b9a4-505982b1f1ee\" (UID: \"3108bb35-3f80-4f5b-b9a4-505982b1f1ee\") "
Oct 30 19:34:23.444265 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.417265    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-maas-api-tls\") pod \"3108bb35-3f80-4f5b-b9a4-505982b1f1ee\" (UID: \"3108bb35-3f80-4f5b-b9a4-505982b1f1ee\") "
Oct 30 19:34:23.444265 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.427942    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-metrics-certs" pod "3108bb35-3f80-4f5b-b9a4-505982b1f1ee" (UID: "3108bb35-3f80-4f5b-b9a4-505982b1f1ee"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:34:23.444265 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.439617    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-kube-api-access-sch5l" pod "3108bb35-3f80-4f5b-b9a4-505982b1f1ee" (UID: "3108bb35-3f80-4f5b-b9a4-505982b1f1ee"). InnerVolumeSpecName "kube-api-access-sch5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:23.452918 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.449989    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-openshift-service-ca" pod "3108bb35-3f80-4f5b-b9a4-505982b1f1ee" (UID: "3108bb35-3f80-4f5b-b9a4-505982b1f1ee"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:34:23.481423 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.472306    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-maas-api-tls" pod "3108bb35-3f80-4f5b-b9a4-505982b1f1ee" (UID: "3108bb35-3f80-4f5b-b9a4-505982b1f1ee"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:34:23.548569 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.548388    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"]
Oct 30 19:34:23.556572 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.554183    4524 generic.go:356] "Generic (PLEG): container finished" podID="3108bb35-3f80-4f5b-b9a4-505982b1f1ee" containerID="a54040016ab4f8da8c7b4919c5e742698bbe973cfa7d663e4e2f98a4a709c073" exitCode=0
Oct 30 19:34:23.556572 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.554489    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"
Oct 30 19:34:23.564191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.561902    4524 cpu_manager.go:414] "containerMap: removing container" podUID="3108bb35-3f80-4f5b-b9a4-505982b1f1ee" containerName="maas-api"
Oct 30 19:34:23.564191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.561926    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3108bb35-3f80-4f5b-b9a4-505982b1f1ee" containerName="maas-api"
Oct 30 19:34:23.564191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.562111    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="3108bb35-3f80-4f5b-b9a4-505982b1f1ee" containerName="maas-api"
Oct 30 19:34:23.564191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.562711    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:23.564191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.562737    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:23.564191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.562751    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sch5l\" (UniqueName: \"kubernetes.io/projected/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-kube-api-access-sch5l\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:23.564191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.562766    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3108bb35-3f80-4f5b-b9a4-505982b1f1ee-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:23.573710 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.573690    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"]
Oct 30 19:34:23.573883 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.573866    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh" event={"ID":"3108bb35-3f80-4f5b-b9a4-505982b1f1ee","Type":"ContainerDied","Data":"a54040016ab4f8da8c7b4919c5e742698bbe973cfa7d663e4e2f98a4a709c073"}
Oct 30 19:34:23.573992 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.573979    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh" event={"ID":"3108bb35-3f80-4f5b-b9a4-505982b1f1ee","Type":"ContainerDied","Data":"faba862a06ab7a2c26d6bdd0f1be199b99a96385d7a59d66147c8d8dfdc2b10a"}
Oct 30 19:34:23.574094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.574083    4524 scope.go:122] "RemoveContainer" containerID="a54040016ab4f8da8c7b4919c5e742698bbe973cfa7d663e4e2f98a4a709c073"
Oct 30 19:34:23.574408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.574392    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.583046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.582234    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-c86696-openshift-default-dockercfg-zhj6j\""
Oct 30 19:34:23.596421 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.596403    4524 scope.go:122] "RemoveContainer" containerID="a54040016ab4f8da8c7b4919c5e742698bbe973cfa7d663e4e2f98a4a709c073"
Oct 30 19:34:23.596743 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:34:23.596724    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a54040016ab4f8da8c7b4919c5e742698bbe973cfa7d663e4e2f98a4a709c073\": container with ID starting with a54040016ab4f8da8c7b4919c5e742698bbe973cfa7d663e4e2f98a4a709c073 not found: ID does not exist" containerID="a54040016ab4f8da8c7b4919c5e742698bbe973cfa7d663e4e2f98a4a709c073"
Oct 30 19:34:23.596798 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.596750    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a54040016ab4f8da8c7b4919c5e742698bbe973cfa7d663e4e2f98a4a709c073"} err="failed to get container status \"a54040016ab4f8da8c7b4919c5e742698bbe973cfa7d663e4e2f98a4a709c073\": rpc error: code = NotFound desc = could not find container \"a54040016ab4f8da8c7b4919c5e742698bbe973cfa7d663e4e2f98a4a709c073\": container with ID starting with a54040016ab4f8da8c7b4919c5e742698bbe973cfa7d663e4e2f98a4a709c073 not found: ID does not exist"
Oct 30 19:34:23.636743 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.635586    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"]
Oct 30 19:34:23.643036 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.643010    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ef5dca-6b98f699dd-4psrh"]
Oct 30 19:34:23.664443 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.664414    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-workload-certs\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.664567 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.664518    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/968cf343-2e3b-4186-875d-2275e313a4b3-istio-token\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.664635 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.664560    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/968cf343-2e3b-4186-875d-2275e313a4b3-istio-podinfo\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.664635 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.664594    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-istio-envoy\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.664761 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.664729    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-istio-data\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.664868 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.664850    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-credential-socket\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.665118 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.665101    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hscs\" (UniqueName: \"kubernetes.io/projected/968cf343-2e3b-4186-875d-2275e313a4b3-kube-api-access-6hscs\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.665160 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.665133    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-workload-socket\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.665232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.665180    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/968cf343-2e3b-4186-875d-2275e313a4b3-istiod-ca-cert\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.766525 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.766494    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hscs\" (UniqueName: \"kubernetes.io/projected/968cf343-2e3b-4186-875d-2275e313a4b3-kube-api-access-6hscs\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.766525 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.766532    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-workload-socket\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.767047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.766562    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/968cf343-2e3b-4186-875d-2275e313a4b3-istiod-ca-cert\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.767047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.766593    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-workload-certs\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.767047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.766621    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/968cf343-2e3b-4186-875d-2275e313a4b3-istio-token\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.767047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.766646    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/968cf343-2e3b-4186-875d-2275e313a4b3-istio-podinfo\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.767047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.766689    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-istio-envoy\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.767047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.766735    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-istio-data\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.767047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.766817    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-credential-socket\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.767047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.766992    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-workload-socket\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.767047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.767023    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-workload-certs\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.767504 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.767249    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-istio-data\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.767504 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.767452    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-credential-socket\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.767619 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.767596    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/968cf343-2e3b-4186-875d-2275e313a4b3-istiod-ca-cert\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.769379 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.769358    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/968cf343-2e3b-4186-875d-2275e313a4b3-istio-podinfo\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.769832 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.769807    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-istio-envoy\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.778626 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.778578    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/968cf343-2e3b-4186-875d-2275e313a4b3-istio-token\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.780214 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.780191    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hscs\" (UniqueName: \"kubernetes.io/projected/968cf343-2e3b-4186-875d-2275e313a4b3-kube-api-access-6hscs\") pod \"e2e-inf-b-c86696-openshift-default-859496475c-9ttpm\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") " pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:23.896800 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:23.896768    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:24.247942 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:24.247844    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"]
Oct 30 19:34:24.257557 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:24.257524    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:34:24.257649 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:24.257594    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:34:24.257649 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:24.257625    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:34:24.392867 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:24.392774    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3108bb35-3f80-4f5b-b9a4-505982b1f1ee" path="/var/lib/kubelet/pods/3108bb35-3f80-4f5b-b9a4-505982b1f1ee/volumes"
Oct 30 19:34:24.567407 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:24.567327    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm" event={"ID":"968cf343-2e3b-4186-875d-2275e313a4b3","Type":"ContainerStarted","Data":"a378ec039dfab88492222047cf2b85b380407f2b2f0da3f70e915a4373a105ef"}
Oct 30 19:34:24.567407 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:24.567385    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm" event={"ID":"968cf343-2e3b-4186-875d-2275e313a4b3","Type":"ContainerStarted","Data":"c4f804ef6cf5a97ea6dd2ba345d6684b1c3795b9d9d7d8b5e662767688b768b7"}
Oct 30 19:34:24.606393 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:24.606342    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm" podStartSLOduration=3.606326993 podStartE2EDuration="3.606326993s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:34:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:34:24.600227849 +0000 UTC m=+1696.341218929" watchObservedRunningTime="2026-10-30 19:34:24.606326993 +0000 UTC m=+1696.347318072"
Oct 30 19:34:24.897264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:24.897229    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:25.899572 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:25.899534    4524 patch_prober.go:29] interesting pod/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused" start-of-body=
Oct 30 19:34:25.900023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:25.899597    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm" podUID="968cf343-2e3b-4186-875d-2275e313a4b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused"
Oct 30 19:34:26.235576 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:26.235479    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q" podUID="23ed05e1-299a-4a05-b555-8a2966d04280" containerName="payload-pre-processing" containerID="cri-o://bd706f3c0881ba7f65b83c0b75ee477533aab86444caf1c6a42d846ac1dca34b" gracePeriod=25
Oct 30 19:34:26.904193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:26.903271    4524 patch_prober.go:29] interesting pod/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused" start-of-body=
Oct 30 19:34:26.904193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:26.903385    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm" podUID="968cf343-2e3b-4186-875d-2275e313a4b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused"
Oct 30 19:34:27.900193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:27.899204    4524 patch_prober.go:29] interesting pod/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused" start-of-body=
Oct 30 19:34:27.900193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:27.899265    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm" podUID="968cf343-2e3b-4186-875d-2275e313a4b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused"
Oct 30 19:34:28.897679 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:28.897643    4524 patch_prober.go:29] interesting pod/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused" start-of-body=
Oct 30 19:34:28.898312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:28.898269    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm" podUID="968cf343-2e3b-4186-875d-2275e313a4b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused"
Oct 30 19:34:29.897621 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:29.897124    4524 patch_prober.go:29] interesting pod/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused" start-of-body=
Oct 30 19:34:29.897621 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:29.897228    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm" podUID="968cf343-2e3b-4186-875d-2275e313a4b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused"
Oct 30 19:34:30.905366 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:30.905316    4524 patch_prober.go:29] interesting pod/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused" start-of-body=
Oct 30 19:34:30.931202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:30.927288    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm" podUID="968cf343-2e3b-4186-875d-2275e313a4b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.147:15021/healthz/ready\": dial tcp 10.133.0.147:15021: connect: connection refused"
Oct 30 19:34:31.996034 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:31.996001    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:31.998031 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:31.998000    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:31.998967 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:31.998948    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:34:32.018429 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:32.018393    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q" podUID="23ed05e1-299a-4a05-b555-8a2966d04280" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:34:32.659190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:32.651154    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"]
Oct 30 19:34:32.659190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:32.651904    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw" podUID="0c3eeede-eb3c-4469-9b85-279c56d473ff" containerName="istio-proxy" containerID="cri-o://d79c4e372d503938a6c524b58c08f5ae0c9e450909bb2bcf845e6f73991b8319" gracePeriod=30
Oct 30 19:34:34.955577 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:34.955538    4524 patch_prober.go:29] interesting pod/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:34:34.955967 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:34.955607    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw" podUID="0c3eeede-eb3c-4469-9b85-279c56d473ff" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:34:37.438160 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:37.438116    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29"]
Oct 30 19:34:38.206073 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.206039    4524 generic.go:356] "Generic (PLEG): container finished" podID="0c3eeede-eb3c-4469-9b85-279c56d473ff" containerID="d79c4e372d503938a6c524b58c08f5ae0c9e450909bb2bcf845e6f73991b8319" exitCode=0
Oct 30 19:34:38.206519 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.206083    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw" event={"ID":"0c3eeede-eb3c-4469-9b85-279c56d473ff","Type":"ContainerDied","Data":"d79c4e372d503938a6c524b58c08f5ae0c9e450909bb2bcf845e6f73991b8319"}
Oct 30 19:34:38.206716 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.206694    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw" event={"ID":"0c3eeede-eb3c-4469-9b85-279c56d473ff","Type":"ContainerDied","Data":"004489f36637f4b0b80ae7fa824539c794588a179e70dd80229749d5d9860406"}
Oct 30 19:34:38.206849 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.206835    4524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="004489f36637f4b0b80ae7fa824539c794588a179e70dd80229749d5d9860406"
Oct 30 19:34:38.211580 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.211563    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:34:38.340064 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.340032    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-podinfo\") pod \"0c3eeede-eb3c-4469-9b85-279c56d473ff\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") "
Oct 30 19:34:38.340281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.340122    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-data\") pod \"0c3eeede-eb3c-4469-9b85-279c56d473ff\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") "
Oct 30 19:34:38.340281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.340209    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-workload-certs\") pod \"0c3eeede-eb3c-4469-9b85-279c56d473ff\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") "
Oct 30 19:34:38.340281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.340240    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c3eeede-eb3c-4469-9b85-279c56d473ff-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c3eeede-eb3c-4469-9b85-279c56d473ff-istiod-ca-cert\") pod \"0c3eeede-eb3c-4469-9b85-279c56d473ff\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") "
Oct 30 19:34:38.340465 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.340315    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c3eeede-eb3c-4469-9b85-279c56d473ff-kube-api-access-xwwck\" (UniqueName: \"kubernetes.io/projected/0c3eeede-eb3c-4469-9b85-279c56d473ff-kube-api-access-xwwck\") pod \"0c3eeede-eb3c-4469-9b85-279c56d473ff\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") "
Oct 30 19:34:38.340465 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.340419    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-credential-socket\") pod \"0c3eeede-eb3c-4469-9b85-279c56d473ff\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") "
Oct 30 19:34:38.340465 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.340432    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-data" pod "0c3eeede-eb3c-4469-9b85-279c56d473ff" (UID: "0c3eeede-eb3c-4469-9b85-279c56d473ff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:38.340609 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.340464    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-workload-certs" pod "0c3eeede-eb3c-4469-9b85-279c56d473ff" (UID: "0c3eeede-eb3c-4469-9b85-279c56d473ff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:38.340609 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.340489    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-envoy\") pod \"0c3eeede-eb3c-4469-9b85-279c56d473ff\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") "
Oct 30 19:34:38.340609 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.340549    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-token\" (UniqueName: \"kubernetes.io/projected/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-token\") pod \"0c3eeede-eb3c-4469-9b85-279c56d473ff\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") "
Oct 30 19:34:38.340747 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.340612    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-workload-socket\") pod \"0c3eeede-eb3c-4469-9b85-279c56d473ff\" (UID: \"0c3eeede-eb3c-4469-9b85-279c56d473ff\") "
Oct 30 19:34:38.340747 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.340688    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-credential-socket" pod "0c3eeede-eb3c-4469-9b85-279c56d473ff" (UID: "0c3eeede-eb3c-4469-9b85-279c56d473ff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:38.340747 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.340710    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c3eeede-eb3c-4469-9b85-279c56d473ff-istiod-ca-cert" pod "0c3eeede-eb3c-4469-9b85-279c56d473ff" (UID: "0c3eeede-eb3c-4469-9b85-279c56d473ff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:34:38.340975 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.340949    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-workload-socket" pod "0c3eeede-eb3c-4469-9b85-279c56d473ff" (UID: "0c3eeede-eb3c-4469-9b85-279c56d473ff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:38.341442 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.341409    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:38.341547 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.341445    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:38.341547 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.341457    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:38.341547 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.341467    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:38.341547 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.341476    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c3eeede-eb3c-4469-9b85-279c56d473ff-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:38.342361 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.342337    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-podinfo" pod "0c3eeede-eb3c-4469-9b85-279c56d473ff" (UID: "0c3eeede-eb3c-4469-9b85-279c56d473ff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:34:38.342602 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.342575    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-token" pod "0c3eeede-eb3c-4469-9b85-279c56d473ff" (UID: "0c3eeede-eb3c-4469-9b85-279c56d473ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:38.342722 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.342703    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-envoy" pod "0c3eeede-eb3c-4469-9b85-279c56d473ff" (UID: "0c3eeede-eb3c-4469-9b85-279c56d473ff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:38.343475 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.343429    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c3eeede-eb3c-4469-9b85-279c56d473ff-kube-api-access-xwwck" pod "0c3eeede-eb3c-4469-9b85-279c56d473ff" (UID: "0c3eeede-eb3c-4469-9b85-279c56d473ff"). InnerVolumeSpecName "kube-api-access-xwwck". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:38.443268 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.443239    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwwck\" (UniqueName: \"kubernetes.io/projected/0c3eeede-eb3c-4469-9b85-279c56d473ff-kube-api-access-xwwck\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:38.443268 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.443268    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:38.443628 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.443282    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:38.443628 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:38.443294    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c3eeede-eb3c-4469-9b85-279c56d473ff-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:39.224481 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:39.224446    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"
Oct 30 19:34:39.257772 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:39.257735    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"]
Oct 30 19:34:39.265578 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:39.265550    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-ef5dca-openshift-default-b7b5597b9-bb9tw"]
Oct 30 19:34:40.395402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:40.395367    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c3eeede-eb3c-4469-9b85-279c56d473ff" path="/var/lib/kubelet/pods/0c3eeede-eb3c-4469-9b85-279c56d473ff/volumes"
Oct 30 19:34:42.018400 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:42.018361    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q" podUID="23ed05e1-299a-4a05-b555-8a2966d04280" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:34:42.439315 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:42.439264    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29" podUID="0662c903-51a8-4e05-a90d-f3baf5488511" containerName="payload-processing" containerID="cri-o://69f864fbf3d4bfecde11a3523fa3d60c50e38888bc29cf06b2c101c4a9cb1152" gracePeriod=25
Oct 30 19:34:43.548087 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:43.548050    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29" podUID="0662c903-51a8-4e05-a90d-f3baf5488511" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:34:45.658933 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:34:45.658892    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:34:46.535961 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:46.535878    4524 generic.go:356] "Generic (PLEG): container finished" podID="23ed05e1-299a-4a05-b555-8a2966d04280" containerID="bd706f3c0881ba7f65b83c0b75ee477533aab86444caf1c6a42d846ac1dca34b" exitCode=0
Oct 30 19:34:46.562467 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:46.562424    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q" event={"ID":"23ed05e1-299a-4a05-b555-8a2966d04280","Type":"ContainerDied","Data":"bd706f3c0881ba7f65b83c0b75ee477533aab86444caf1c6a42d846ac1dca34b"}
Oct 30 19:34:46.723564 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:46.723536    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q"
Oct 30 19:34:46.866747 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:46.866708    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/23ed05e1-299a-4a05-b555-8a2966d04280-kube-api-access-f5m2w\" (UniqueName: \"kubernetes.io/projected/23ed05e1-299a-4a05-b555-8a2966d04280-kube-api-access-f5m2w\") pod \"23ed05e1-299a-4a05-b555-8a2966d04280\" (UID: \"23ed05e1-299a-4a05-b555-8a2966d04280\") "
Oct 30 19:34:46.866955 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:46.866807    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/23ed05e1-299a-4a05-b555-8a2966d04280-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23ed05e1-299a-4a05-b555-8a2966d04280-plugins-config-volume\") pod \"23ed05e1-299a-4a05-b555-8a2966d04280\" (UID: \"23ed05e1-299a-4a05-b555-8a2966d04280\") "
Oct 30 19:34:46.867197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:46.867145    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23ed05e1-299a-4a05-b555-8a2966d04280-plugins-config-volume" pod "23ed05e1-299a-4a05-b555-8a2966d04280" (UID: "23ed05e1-299a-4a05-b555-8a2966d04280"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:34:46.867618 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:46.867598    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/23ed05e1-299a-4a05-b555-8a2966d04280-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:46.869046 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:46.869021    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23ed05e1-299a-4a05-b555-8a2966d04280-kube-api-access-f5m2w" pod "23ed05e1-299a-4a05-b555-8a2966d04280" (UID: "23ed05e1-299a-4a05-b555-8a2966d04280"). InnerVolumeSpecName "kube-api-access-f5m2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:46.968788 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:46.968749    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5m2w\" (UniqueName: \"kubernetes.io/projected/23ed05e1-299a-4a05-b555-8a2966d04280-kube-api-access-f5m2w\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:47.549870 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:47.549828    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q"
Oct 30 19:34:47.550105 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:47.549967    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q" event={"ID":"23ed05e1-299a-4a05-b555-8a2966d04280","Type":"ContainerDied","Data":"9d7478c0783653c4a68195be6b63eccbe5abacc8296efd89e0dfdb6801a45211"}
Oct 30 19:34:47.550105 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:47.550019    4524 scope.go:122] "RemoveContainer" containerID="bd706f3c0881ba7f65b83c0b75ee477533aab86444caf1c6a42d846ac1dca34b"
Oct 30 19:34:47.575578 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:47.575547    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q"]
Oct 30 19:34:47.579994 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:47.579971    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-ef5dca-669d6c55f5-gzf2q"]
Oct 30 19:34:47.916826 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:47.916791    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"]
Oct 30 19:34:47.917371 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:47.917063    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm" podUID="4772e6be-0a1a-4def-b2fa-c4f09f7df3a3" containerName="maas-api" containerID="cri-o://c8a1662c12c34bc51ef7769c12adac05796d87a567b0c72d4fa46f02224c5d6c" gracePeriod=30
Oct 30 19:34:48.638662 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:48.638630    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="23ed05e1-299a-4a05-b555-8a2966d04280" path="/var/lib/kubelet/pods/23ed05e1-299a-4a05-b555-8a2966d04280/volumes"
Oct 30 19:34:48.643762 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:48.643732    4524 generic.go:356] "Generic (PLEG): container finished" podID="4772e6be-0a1a-4def-b2fa-c4f09f7df3a3" containerID="c8a1662c12c34bc51ef7769c12adac05796d87a567b0c72d4fa46f02224c5d6c" exitCode=0
Oct 30 19:34:48.644127 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:48.644106    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm" event={"ID":"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3","Type":"ContainerDied","Data":"c8a1662c12c34bc51ef7769c12adac05796d87a567b0c72d4fa46f02224c5d6c"}
Oct 30 19:34:48.644364 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:48.644346    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm" event={"ID":"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3","Type":"ContainerDied","Data":"319e1888be437c45d46a84a981cc9fb9ad1347928503c2214b897dbb94ee17ac"}
Oct 30 19:34:48.644490 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:48.644477    4524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="319e1888be437c45d46a84a981cc9fb9ad1347928503c2214b897dbb94ee17ac"
Oct 30 19:34:48.648353 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:48.648330    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"
Oct 30 19:34:48.732231 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:48.732200    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-metrics-certs\") pod \"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3\" (UID: \"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3\") "
Oct 30 19:34:48.732427 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:48.732271    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-kube-api-access-8lsmz\" (UniqueName: \"kubernetes.io/projected/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-kube-api-access-8lsmz\") pod \"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3\" (UID: \"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3\") "
Oct 30 19:34:48.732427 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:48.732302    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-maas-api-tls\") pod \"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3\" (UID: \"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3\") "
Oct 30 19:34:48.732427 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:48.732406    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-openshift-service-ca\") pod \"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3\" (UID: \"4772e6be-0a1a-4def-b2fa-c4f09f7df3a3\") "
Oct 30 19:34:48.733258 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:48.733225    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-openshift-service-ca" pod "4772e6be-0a1a-4def-b2fa-c4f09f7df3a3" (UID: "4772e6be-0a1a-4def-b2fa-c4f09f7df3a3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:34:48.735251 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:48.735221    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-maas-api-tls" pod "4772e6be-0a1a-4def-b2fa-c4f09f7df3a3" (UID: "4772e6be-0a1a-4def-b2fa-c4f09f7df3a3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:34:48.736928 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:48.736897    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-kube-api-access-8lsmz" pod "4772e6be-0a1a-4def-b2fa-c4f09f7df3a3" (UID: "4772e6be-0a1a-4def-b2fa-c4f09f7df3a3"). InnerVolumeSpecName "kube-api-access-8lsmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:48.738575 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:48.738554    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-metrics-certs" pod "4772e6be-0a1a-4def-b2fa-c4f09f7df3a3" (UID: "4772e6be-0a1a-4def-b2fa-c4f09f7df3a3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:34:48.865319 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:48.865004    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:48.865319 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:48.865047    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lsmz\" (UniqueName: \"kubernetes.io/projected/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-kube-api-access-8lsmz\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:48.865319 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:48.865080    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:48.865319 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:48.865095    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:49.662187 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:49.662136    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"
Oct 30 19:34:49.693142 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:49.693110    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"]
Oct 30 19:34:49.697808 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:49.697779    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-afdf16-cbc85b548-ksdkm"]
Oct 30 19:34:50.395463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:50.395422    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4772e6be-0a1a-4def-b2fa-c4f09f7df3a3" path="/var/lib/kubelet/pods/4772e6be-0a1a-4def-b2fa-c4f09f7df3a3/volumes"
Oct 30 19:34:51.829193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:51.820598    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"]
Oct 30 19:34:51.829193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:51.820809    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g" podUID="0e1416a9-3d41-4166-bc0a-01dbc28aad2d" containerName="istio-proxy" containerID="cri-o://7e6640edd3830652832478998218c506a60327e2b82e3d167e54510cb88e37d5" gracePeriod=30
Oct 30 19:34:51.829193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:51.829038    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"]
Oct 30 19:34:51.829789 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:51.829332    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp" podUID="68953aed-0987-40c8-932c-fa4591f71831" containerName="istio-proxy" containerID="cri-o://c7445b8c4b6709aa85d73292565f09c31d7b4777baba0d05d4523c95baf2fce5" gracePeriod=30
Oct 30 19:34:51.915616 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:51.915581    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"]
Oct 30 19:34:51.915855 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:51.915826    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6" podUID="1d63b36f-3cfa-4438-8fb1-21480c9e9527" containerName="istio-proxy" containerID="cri-o://4ab11722d99de13c19ffed2c9c210735bdc599ea12cd356ac3ccb191ab7dadfb" gracePeriod=30
Oct 30 19:34:53.547971 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:53.547931    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29" podUID="0662c903-51a8-4e05-a90d-f3baf5488511" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:34:57.356208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.342645    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:34:57.406072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.404826    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-envoy\") pod \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") "
Oct 30 19:34:57.406072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.404875    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-token\" (UniqueName: \"kubernetes.io/projected/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-token\") pod \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") "
Oct 30 19:34:57.406072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.404906    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-podinfo\") pod \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") "
Oct 30 19:34:57.406072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.404943    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-data\") pod \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") "
Oct 30 19:34:57.406072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.405075    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d63b36f-3cfa-4438-8fb1-21480c9e9527-kube-api-access-7cd4t\" (UniqueName: \"kubernetes.io/projected/1d63b36f-3cfa-4438-8fb1-21480c9e9527-kube-api-access-7cd4t\") pod \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") "
Oct 30 19:34:57.406072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.405201    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-credential-socket\") pod \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") "
Oct 30 19:34:57.406072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.405266    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-workload-socket\") pod \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") "
Oct 30 19:34:57.406072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.405314    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istiod-ca-cert\") pod \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") "
Oct 30 19:34:57.406072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.405340    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-workload-certs\") pod \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\" (UID: \"1d63b36f-3cfa-4438-8fb1-21480c9e9527\") "
Oct 30 19:34:57.410725 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.410377    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-workload-certs" pod "1d63b36f-3cfa-4438-8fb1-21480c9e9527" (UID: "1d63b36f-3cfa-4438-8fb1-21480c9e9527"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:57.412775 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.412427    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-data" pod "1d63b36f-3cfa-4438-8fb1-21480c9e9527" (UID: "1d63b36f-3cfa-4438-8fb1-21480c9e9527"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:57.413087 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.413044    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istiod-ca-cert" pod "1d63b36f-3cfa-4438-8fb1-21480c9e9527" (UID: "1d63b36f-3cfa-4438-8fb1-21480c9e9527"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:34:57.413484 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.413448    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-credential-socket" pod "1d63b36f-3cfa-4438-8fb1-21480c9e9527" (UID: "1d63b36f-3cfa-4438-8fb1-21480c9e9527"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:57.413593 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.413544    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-workload-socket" pod "1d63b36f-3cfa-4438-8fb1-21480c9e9527" (UID: "1d63b36f-3cfa-4438-8fb1-21480c9e9527"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:57.434510 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.434237    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-podinfo" pod "1d63b36f-3cfa-4438-8fb1-21480c9e9527" (UID: "1d63b36f-3cfa-4438-8fb1-21480c9e9527"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:34:57.434510 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.434370    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-token" pod "1d63b36f-3cfa-4438-8fb1-21480c9e9527" (UID: "1d63b36f-3cfa-4438-8fb1-21480c9e9527"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:57.437274 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.437227    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-envoy" pod "1d63b36f-3cfa-4438-8fb1-21480c9e9527" (UID: "1d63b36f-3cfa-4438-8fb1-21480c9e9527"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:57.439371 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.439342    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d63b36f-3cfa-4438-8fb1-21480c9e9527-kube-api-access-7cd4t" pod "1d63b36f-3cfa-4438-8fb1-21480c9e9527" (UID: "1d63b36f-3cfa-4438-8fb1-21480c9e9527"). InnerVolumeSpecName "kube-api-access-7cd4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:57.454614 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.454595    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:34:57.459742 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.459724    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:34:57.510616 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.510576    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/68953aed-0987-40c8-932c-fa4591f71831-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68953aed-0987-40c8-932c-fa4591f71831-istio-podinfo\") pod \"68953aed-0987-40c8-932c-fa4591f71831\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") "
Oct 30 19:34:57.510809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.510668    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-workload-certs\") pod \"68953aed-0987-40c8-932c-fa4591f71831\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") "
Oct 30 19:34:57.510809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.510766    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-kube-api-access-dj7b8\" (UniqueName: \"kubernetes.io/projected/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-kube-api-access-dj7b8\") pod \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") "
Oct 30 19:34:57.510929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.510846    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68953aed-0987-40c8-932c-fa4591f71831-istio-token\" (UniqueName: \"kubernetes.io/projected/68953aed-0987-40c8-932c-fa4591f71831-istio-token\") pod \"68953aed-0987-40c8-932c-fa4591f71831\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") "
Oct 30 19:34:57.510929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.510917    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-podinfo\") pod \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") "
Oct 30 19:34:57.511050 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.510985    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-envoy\") pod \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") "
Oct 30 19:34:57.511050 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.511036    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/68953aed-0987-40c8-932c-fa4591f71831-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68953aed-0987-40c8-932c-fa4591f71831-istiod-ca-cert\") pod \"68953aed-0987-40c8-932c-fa4591f71831\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") "
Oct 30 19:34:57.511161 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.511085    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-token\" (UniqueName: \"kubernetes.io/projected/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-token\") pod \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") "
Oct 30 19:34:57.511161 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.511130    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68953aed-0987-40c8-932c-fa4591f71831-kube-api-access-tvbzd\" (UniqueName: \"kubernetes.io/projected/68953aed-0987-40c8-932c-fa4591f71831-kube-api-access-tvbzd\") pod \"68953aed-0987-40c8-932c-fa4591f71831\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") "
Oct 30 19:34:57.511295 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.511257    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-workload-socket\") pod \"68953aed-0987-40c8-932c-fa4591f71831\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") "
Oct 30 19:34:57.511355 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.511315    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-istio-envoy\") pod \"68953aed-0987-40c8-932c-fa4591f71831\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") "
Oct 30 19:34:57.511401 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.511379    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-data\") pod \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") "
Oct 30 19:34:57.511479 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.511463    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-istio-data\") pod \"68953aed-0987-40c8-932c-fa4591f71831\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") "
Oct 30 19:34:57.511535 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.511505    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-workload-socket\") pod \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") "
Oct 30 19:34:57.511628 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.511613    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-credential-socket\") pod \"68953aed-0987-40c8-932c-fa4591f71831\" (UID: \"68953aed-0987-40c8-932c-fa4591f71831\") "
Oct 30 19:34:57.511670 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.511651    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istiod-ca-cert\") pod \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") "
Oct 30 19:34:57.511789 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.511773    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-credential-socket\") pod \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") "
Oct 30 19:34:57.511848 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.511811    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-workload-certs\") pod \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\" (UID: \"0e1416a9-3d41-4166-bc0a-01dbc28aad2d\") "
Oct 30 19:34:57.514112 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.514066    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.514485 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.514113    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.514485 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.514130    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.514485 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.514146    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.514485 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.514161    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.514485 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.514186    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.514485 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.514200    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cd4t\" (UniqueName: \"kubernetes.io/projected/1d63b36f-3cfa-4438-8fb1-21480c9e9527-kube-api-access-7cd4t\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.514485 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.514216    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.514485 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.514230    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d63b36f-3cfa-4438-8fb1-21480c9e9527-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.514931 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.514780    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-credential-socket" pod "68953aed-0987-40c8-932c-fa4591f71831" (UID: "68953aed-0987-40c8-932c-fa4591f71831"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:57.515109 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.515052    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istiod-ca-cert" pod "0e1416a9-3d41-4166-bc0a-01dbc28aad2d" (UID: "0e1416a9-3d41-4166-bc0a-01dbc28aad2d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:34:57.515562 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.515357    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-credential-socket" pod "0e1416a9-3d41-4166-bc0a-01dbc28aad2d" (UID: "0e1416a9-3d41-4166-bc0a-01dbc28aad2d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:57.515562 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.515511    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-workload-certs" pod "0e1416a9-3d41-4166-bc0a-01dbc28aad2d" (UID: "0e1416a9-3d41-4166-bc0a-01dbc28aad2d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:57.515810 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.515784    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-workload-socket" pod "0e1416a9-3d41-4166-bc0a-01dbc28aad2d" (UID: "0e1416a9-3d41-4166-bc0a-01dbc28aad2d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:57.518714 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.518675    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/68953aed-0987-40c8-932c-fa4591f71831-istio-podinfo" pod "68953aed-0987-40c8-932c-fa4591f71831" (UID: "68953aed-0987-40c8-932c-fa4591f71831"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:34:57.519041 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.519021    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-workload-certs" pod "68953aed-0987-40c8-932c-fa4591f71831" (UID: "68953aed-0987-40c8-932c-fa4591f71831"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:57.535721 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.525301    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-workload-socket" pod "68953aed-0987-40c8-932c-fa4591f71831" (UID: "68953aed-0987-40c8-932c-fa4591f71831"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:57.536864 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.536568    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-token" pod "0e1416a9-3d41-4166-bc0a-01dbc28aad2d" (UID: "0e1416a9-3d41-4166-bc0a-01dbc28aad2d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:57.536864 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.536841    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68953aed-0987-40c8-932c-fa4591f71831-istio-token" pod "68953aed-0987-40c8-932c-fa4591f71831" (UID: "68953aed-0987-40c8-932c-fa4591f71831"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:57.537614 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.537589    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-istio-envoy" pod "68953aed-0987-40c8-932c-fa4591f71831" (UID: "68953aed-0987-40c8-932c-fa4591f71831"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:57.537737 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.537718    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68953aed-0987-40c8-932c-fa4591f71831-istiod-ca-cert" pod "68953aed-0987-40c8-932c-fa4591f71831" (UID: "68953aed-0987-40c8-932c-fa4591f71831"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:34:57.537959 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.537940    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-istio-data" pod "68953aed-0987-40c8-932c-fa4591f71831" (UID: "68953aed-0987-40c8-932c-fa4591f71831"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:57.538113 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.538088    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-data" pod "0e1416a9-3d41-4166-bc0a-01dbc28aad2d" (UID: "0e1416a9-3d41-4166-bc0a-01dbc28aad2d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:57.538456 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.538436    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-podinfo" pod "0e1416a9-3d41-4166-bc0a-01dbc28aad2d" (UID: "0e1416a9-3d41-4166-bc0a-01dbc28aad2d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:34:57.538820 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.538798    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-kube-api-access-dj7b8" pod "0e1416a9-3d41-4166-bc0a-01dbc28aad2d" (UID: "0e1416a9-3d41-4166-bc0a-01dbc28aad2d"). InnerVolumeSpecName "kube-api-access-dj7b8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:57.539264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.539231    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68953aed-0987-40c8-932c-fa4591f71831-kube-api-access-tvbzd" pod "68953aed-0987-40c8-932c-fa4591f71831" (UID: "68953aed-0987-40c8-932c-fa4591f71831"). InnerVolumeSpecName "kube-api-access-tvbzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:57.540795 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.540675    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-envoy" pod "0e1416a9-3d41-4166-bc0a-01dbc28aad2d" (UID: "0e1416a9-3d41-4166-bc0a-01dbc28aad2d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:57.616094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.615805    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dj7b8\" (UniqueName: \"kubernetes.io/projected/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-kube-api-access-dj7b8\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.616094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.615852    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68953aed-0987-40c8-932c-fa4591f71831-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.616094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.615867    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.616094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.615881    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.616094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.615895    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68953aed-0987-40c8-932c-fa4591f71831-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.616094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.615908    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.616094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.615922    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvbzd\" (UniqueName: \"kubernetes.io/projected/68953aed-0987-40c8-932c-fa4591f71831-kube-api-access-tvbzd\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.616094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.615935    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.616094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.615949    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.616094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.615961    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.616094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.615976    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.616094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.615989    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.616094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.616002    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.616094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.616015    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.616094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.616029    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.616094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.616042    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e1416a9-3d41-4166-bc0a-01dbc28aad2d-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.616094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.616056    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68953aed-0987-40c8-932c-fa4591f71831-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.616094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.616070    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68953aed-0987-40c8-932c-fa4591f71831-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:57.808246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.796221    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"]
Oct 30 19:34:57.808246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.797521    4524 cpu_manager.go:414] "containerMap: removing container" podUID="4772e6be-0a1a-4def-b2fa-c4f09f7df3a3" containerName="maas-api"
Oct 30 19:34:57.808246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.797544    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4772e6be-0a1a-4def-b2fa-c4f09f7df3a3" containerName="maas-api"
Oct 30 19:34:57.808246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.797557    4524 cpu_manager.go:414] "containerMap: removing container" podUID="23ed05e1-299a-4a05-b555-8a2966d04280" containerName="payload-pre-processing"
Oct 30 19:34:57.808246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.797566    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23ed05e1-299a-4a05-b555-8a2966d04280" containerName="payload-pre-processing"
Oct 30 19:34:57.808246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.797622    4524 cpu_manager.go:414] "containerMap: removing container" podUID="0c3eeede-eb3c-4469-9b85-279c56d473ff" containerName="istio-proxy"
Oct 30 19:34:57.808246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.797630    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c3eeede-eb3c-4469-9b85-279c56d473ff" containerName="istio-proxy"
Oct 30 19:34:57.808246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.797645    4524 cpu_manager.go:414] "containerMap: removing container" podUID="68953aed-0987-40c8-932c-fa4591f71831" containerName="istio-proxy"
Oct 30 19:34:57.808246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.797653    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68953aed-0987-40c8-932c-fa4591f71831" containerName="istio-proxy"
Oct 30 19:34:57.808246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.797669    4524 cpu_manager.go:414] "containerMap: removing container" podUID="1d63b36f-3cfa-4438-8fb1-21480c9e9527" containerName="istio-proxy"
Oct 30 19:34:57.808246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.797677    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d63b36f-3cfa-4438-8fb1-21480c9e9527" containerName="istio-proxy"
Oct 30 19:34:57.808246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.797690    4524 cpu_manager.go:414] "containerMap: removing container" podUID="0e1416a9-3d41-4166-bc0a-01dbc28aad2d" containerName="istio-proxy"
Oct 30 19:34:57.808246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.797697    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e1416a9-3d41-4166-bc0a-01dbc28aad2d" containerName="istio-proxy"
Oct 30 19:34:57.808246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.797883    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="4772e6be-0a1a-4def-b2fa-c4f09f7df3a3" containerName="maas-api"
Oct 30 19:34:57.808246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.797899    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="23ed05e1-299a-4a05-b555-8a2966d04280" containerName="payload-pre-processing"
Oct 30 19:34:57.808246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.797916    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c3eeede-eb3c-4469-9b85-279c56d473ff" containerName="istio-proxy"
Oct 30 19:34:57.808246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.797926    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="68953aed-0987-40c8-932c-fa4591f71831" containerName="istio-proxy"
Oct 30 19:34:57.808246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.797937    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d63b36f-3cfa-4438-8fb1-21480c9e9527" containerName="istio-proxy"
Oct 30 19:34:57.808246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.797947    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e1416a9-3d41-4166-bc0a-01dbc28aad2d" containerName="istio-proxy"
Oct 30 19:34:57.846198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.833973    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"
Oct 30 19:34:57.847394 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.847364    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-1fa500-serving-cert\""
Oct 30 19:34:57.865849 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.865808    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"]
Oct 30 19:34:57.996436 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.991461    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6eecd016-5523-46a5-919b-09e0772cba31-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s\" (UID: \"6eecd016-5523-46a5-919b-09e0772cba31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"
Oct 30 19:34:57.996436 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.992563    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l728v\" (UniqueName: \"kubernetes.io/projected/6eecd016-5523-46a5-919b-09e0772cba31-kube-api-access-l728v\") pod \"maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s\" (UID: \"6eecd016-5523-46a5-919b-09e0772cba31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"
Oct 30 19:34:57.996436 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.992647    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eecd016-5523-46a5-919b-09e0772cba31-metrics-certs\") pod \"maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s\" (UID: \"6eecd016-5523-46a5-919b-09e0772cba31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"
Oct 30 19:34:57.996436 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:57.992778    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6eecd016-5523-46a5-919b-09e0772cba31-maas-api-tls\") pod \"maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s\" (UID: \"6eecd016-5523-46a5-919b-09e0772cba31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"
Oct 30 19:34:58.155230 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.145389    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6eecd016-5523-46a5-919b-09e0772cba31-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s\" (UID: \"6eecd016-5523-46a5-919b-09e0772cba31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"
Oct 30 19:34:58.155519 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.146460    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6eecd016-5523-46a5-919b-09e0772cba31-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s\" (UID: \"6eecd016-5523-46a5-919b-09e0772cba31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"
Oct 30 19:34:58.155731 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.153119    4524 generic.go:356] "Generic (PLEG): container finished" podID="1d63b36f-3cfa-4438-8fb1-21480c9e9527" containerID="4ab11722d99de13c19ffed2c9c210735bdc599ea12cd356ac3ccb191ab7dadfb" exitCode=0
Oct 30 19:34:58.156084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.153146    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6" event={"ID":"1d63b36f-3cfa-4438-8fb1-21480c9e9527","Type":"ContainerDied","Data":"4ab11722d99de13c19ffed2c9c210735bdc599ea12cd356ac3ccb191ab7dadfb"}
Oct 30 19:34:58.156194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.156151    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6" event={"ID":"1d63b36f-3cfa-4438-8fb1-21480c9e9527","Type":"ContainerDied","Data":"e3140b4172c423843d76347bc99b5a5ef10f610d3449aee6c0f3481f00eedb67"}
Oct 30 19:34:58.156259 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.155673    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l728v\" (UniqueName: \"kubernetes.io/projected/6eecd016-5523-46a5-919b-09e0772cba31-kube-api-access-l728v\") pod \"maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s\" (UID: \"6eecd016-5523-46a5-919b-09e0772cba31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"
Oct 30 19:34:58.156314 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.156300    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eecd016-5523-46a5-919b-09e0772cba31-metrics-certs\") pod \"maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s\" (UID: \"6eecd016-5523-46a5-919b-09e0772cba31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"
Oct 30 19:34:58.156376 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.156359    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6eecd016-5523-46a5-919b-09e0772cba31-maas-api-tls\") pod \"maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s\" (UID: \"6eecd016-5523-46a5-919b-09e0772cba31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"
Oct 30 19:34:58.156506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.156493    4524 scope.go:122] "RemoveContainer" containerID="4ab11722d99de13c19ffed2c9c210735bdc599ea12cd356ac3ccb191ab7dadfb"
Oct 30 19:34:58.157346 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:34:58.156569    4524 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-serving-cert: secret "maas-api-e2e-inf-b-1fa500-serving-cert" not found
Oct 30 19:34:58.157435 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:34:58.157404    4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eecd016-5523-46a5-919b-09e0772cba31-maas-api-tls podName:6eecd016-5523-46a5-919b-09e0772cba31 nodeName:}" failed. No retries permitted until 2026-10-30 19:34:58.657382583 +0000 UTC m=+1730.398373654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6eecd016-5523-46a5-919b-09e0772cba31-maas-api-tls") pod "maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s" (UID: "6eecd016-5523-46a5-919b-09e0772cba31") : secret "maas-api-e2e-inf-b-1fa500-serving-cert" not found
Oct 30 19:34:58.157642 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.153281    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"
Oct 30 19:34:58.236680 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.236644    4524 generic.go:356] "Generic (PLEG): container finished" podID="0e1416a9-3d41-4166-bc0a-01dbc28aad2d" containerID="7e6640edd3830652832478998218c506a60327e2b82e3d167e54510cb88e37d5" exitCode=0
Oct 30 19:34:58.236922 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.236903    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g" event={"ID":"0e1416a9-3d41-4166-bc0a-01dbc28aad2d","Type":"ContainerDied","Data":"7e6640edd3830652832478998218c506a60327e2b82e3d167e54510cb88e37d5"}
Oct 30 19:34:58.237001 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.236961    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g" event={"ID":"0e1416a9-3d41-4166-bc0a-01dbc28aad2d","Type":"ContainerDied","Data":"699474c6f5578ff9d96438bd66a54ae59b6560f63c3551516dd48574995ba036"}
Oct 30 19:34:58.238754 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.238732    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eecd016-5523-46a5-919b-09e0772cba31-metrics-certs\") pod \"maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s\" (UID: \"6eecd016-5523-46a5-919b-09e0772cba31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"
Oct 30 19:34:58.250084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.247326    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"
Oct 30 19:34:58.250084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.247443    4524 generic.go:356] "Generic (PLEG): container finished" podID="68953aed-0987-40c8-932c-fa4591f71831" containerID="c7445b8c4b6709aa85d73292565f09c31d7b4777baba0d05d4523c95baf2fce5" exitCode=0
Oct 30 19:34:58.250084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.247611    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp" event={"ID":"68953aed-0987-40c8-932c-fa4591f71831","Type":"ContainerDied","Data":"c7445b8c4b6709aa85d73292565f09c31d7b4777baba0d05d4523c95baf2fce5"}
Oct 30 19:34:58.250084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.247657    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp" event={"ID":"68953aed-0987-40c8-932c-fa4591f71831","Type":"ContainerDied","Data":"d671fe75c25a1c03bf590563bac434388d009810e978f073e79ae25caa8325e6"}
Oct 30 19:34:58.250084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.247724    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"
Oct 30 19:34:58.250084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.249011    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l728v\" (UniqueName: \"kubernetes.io/projected/6eecd016-5523-46a5-919b-09e0772cba31-kube-api-access-l728v\") pod \"maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s\" (UID: \"6eecd016-5523-46a5-919b-09e0772cba31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"
Oct 30 19:34:58.274440 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.274413    4524 scope.go:122] "RemoveContainer" containerID="4ab11722d99de13c19ffed2c9c210735bdc599ea12cd356ac3ccb191ab7dadfb"
Oct 30 19:34:58.274774 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:34:58.274751    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ab11722d99de13c19ffed2c9c210735bdc599ea12cd356ac3ccb191ab7dadfb\": container with ID starting with 4ab11722d99de13c19ffed2c9c210735bdc599ea12cd356ac3ccb191ab7dadfb not found: ID does not exist" containerID="4ab11722d99de13c19ffed2c9c210735bdc599ea12cd356ac3ccb191ab7dadfb"
Oct 30 19:34:58.274859 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.274783    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ab11722d99de13c19ffed2c9c210735bdc599ea12cd356ac3ccb191ab7dadfb"} err="failed to get container status \"4ab11722d99de13c19ffed2c9c210735bdc599ea12cd356ac3ccb191ab7dadfb\": rpc error: code = NotFound desc = could not find container \"4ab11722d99de13c19ffed2c9c210735bdc599ea12cd356ac3ccb191ab7dadfb\": container with ID starting with 4ab11722d99de13c19ffed2c9c210735bdc599ea12cd356ac3ccb191ab7dadfb not found: ID does not exist"
Oct 30 19:34:58.274859 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.274800    4524 scope.go:122] "RemoveContainer" containerID="7e6640edd3830652832478998218c506a60327e2b82e3d167e54510cb88e37d5"
Oct 30 19:34:58.285245 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.284874    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"]
Oct 30 19:34:58.287919 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.287897    4524 scope.go:122] "RemoveContainer" containerID="7e6640edd3830652832478998218c506a60327e2b82e3d167e54510cb88e37d5"
Oct 30 19:34:58.288325 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:34:58.288296    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e6640edd3830652832478998218c506a60327e2b82e3d167e54510cb88e37d5\": container with ID starting with 7e6640edd3830652832478998218c506a60327e2b82e3d167e54510cb88e37d5 not found: ID does not exist" containerID="7e6640edd3830652832478998218c506a60327e2b82e3d167e54510cb88e37d5"
Oct 30 19:34:58.288447 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.288335    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e6640edd3830652832478998218c506a60327e2b82e3d167e54510cb88e37d5"} err="failed to get container status \"7e6640edd3830652832478998218c506a60327e2b82e3d167e54510cb88e37d5\": rpc error: code = NotFound desc = could not find container \"7e6640edd3830652832478998218c506a60327e2b82e3d167e54510cb88e37d5\": container with ID starting with 7e6640edd3830652832478998218c506a60327e2b82e3d167e54510cb88e37d5 not found: ID does not exist"
Oct 30 19:34:58.288447 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.288357    4524 scope.go:122] "RemoveContainer" containerID="c7445b8c4b6709aa85d73292565f09c31d7b4777baba0d05d4523c95baf2fce5"
Oct 30 19:34:58.290913 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.290892    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-5bmq6"]
Oct 30 19:34:58.298453 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.298429    4524 scope.go:122] "RemoveContainer" containerID="c7445b8c4b6709aa85d73292565f09c31d7b4777baba0d05d4523c95baf2fce5"
Oct 30 19:34:58.298726 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:34:58.298700    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7445b8c4b6709aa85d73292565f09c31d7b4777baba0d05d4523c95baf2fce5\": container with ID starting with c7445b8c4b6709aa85d73292565f09c31d7b4777baba0d05d4523c95baf2fce5 not found: ID does not exist" containerID="c7445b8c4b6709aa85d73292565f09c31d7b4777baba0d05d4523c95baf2fce5"
Oct 30 19:34:58.298792 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.298736    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7445b8c4b6709aa85d73292565f09c31d7b4777baba0d05d4523c95baf2fce5"} err="failed to get container status \"c7445b8c4b6709aa85d73292565f09c31d7b4777baba0d05d4523c95baf2fce5\": rpc error: code = NotFound desc = could not find container \"c7445b8c4b6709aa85d73292565f09c31d7b4777baba0d05d4523c95baf2fce5\": container with ID starting with c7445b8c4b6709aa85d73292565f09c31d7b4777baba0d05d4523c95baf2fce5 not found: ID does not exist"
Oct 30 19:34:58.309454 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.309435    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"]
Oct 30 19:34:58.314156 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.314135    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-2hc5g"]
Oct 30 19:34:58.321976 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.321956    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"]
Oct 30 19:34:58.331630 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.331608    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-vxrdp"]
Oct 30 19:34:58.394005 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.393980    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e1416a9-3d41-4166-bc0a-01dbc28aad2d" path="/var/lib/kubelet/pods/0e1416a9-3d41-4166-bc0a-01dbc28aad2d/volumes"
Oct 30 19:34:58.394545 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.394529    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1d63b36f-3cfa-4438-8fb1-21480c9e9527" path="/var/lib/kubelet/pods/1d63b36f-3cfa-4438-8fb1-21480c9e9527/volumes"
Oct 30 19:34:58.394974 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.394961    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="68953aed-0987-40c8-932c-fa4591f71831" path="/var/lib/kubelet/pods/68953aed-0987-40c8-932c-fa4591f71831/volumes"
Oct 30 19:34:58.663409 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.663380    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6eecd016-5523-46a5-919b-09e0772cba31-maas-api-tls\") pod \"maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s\" (UID: \"6eecd016-5523-46a5-919b-09e0772cba31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"
Oct 30 19:34:58.665911 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.665879    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6eecd016-5523-46a5-919b-09e0772cba31-maas-api-tls\") pod \"maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s\" (UID: \"6eecd016-5523-46a5-919b-09e0772cba31\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"
Oct 30 19:34:58.887648 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:58.887419    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"
Oct 30 19:34:59.598851 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:59.598808    4524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:34:59.615423 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:34:59.615397    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"]
Oct 30 19:35:00.172159 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.172126    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"]
Oct 30 19:35:00.193791 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.193762    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"
Oct 30 19:35:00.195281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.195255    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"]
Oct 30 19:35:00.247273 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.247243    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w6-69c573-serving-cert\""
Oct 30 19:35:00.251994 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.251969    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmjjf\" (UniqueName: \"kubernetes.io/projected/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-kube-api-access-bmjjf\") pod \"maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs\" (UID: \"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"
Oct 30 19:35:00.252084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.252034    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs\" (UID: \"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"
Oct 30 19:35:00.252142 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.252108    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs\" (UID: \"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"
Oct 30 19:35:00.252193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.252145    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs\" (UID: \"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"
Oct 30 19:35:00.358606 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.358513    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmjjf\" (UniqueName: \"kubernetes.io/projected/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-kube-api-access-bmjjf\") pod \"maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs\" (UID: \"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"
Oct 30 19:35:00.359280 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.359255    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs\" (UID: \"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"
Oct 30 19:35:00.360226 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.360203    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs\" (UID: \"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"
Oct 30 19:35:00.360849 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.360821    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs\" (UID: \"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"
Oct 30 19:35:00.361520 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.360123    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs\" (UID: \"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"
Oct 30 19:35:00.378916 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.378885    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs\" (UID: \"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"
Oct 30 19:35:00.380740 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.380612    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs\" (UID: \"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"
Oct 30 19:35:00.395205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.389370    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmjjf\" (UniqueName: \"kubernetes.io/projected/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-kube-api-access-bmjjf\") pod \"maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs\" (UID: \"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"
Oct 30 19:35:00.463344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.463306    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s" event={"ID":"6eecd016-5523-46a5-919b-09e0772cba31","Type":"ContainerStarted","Data":"43ee0bd9ff2db32c4b7b84b353b49acb65927a51351b145138561700132edb6e"}
Oct 30 19:35:00.463550 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.463374    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s" event={"ID":"6eecd016-5523-46a5-919b-09e0772cba31","Type":"ContainerStarted","Data":"de6a3ff2e048f2764264243be91a7eba54b26a444d76426f62f9dda5378c07b6"}
Oct 30 19:35:00.463550 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.463405    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"
Oct 30 19:35:00.490706 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.490647    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s" podStartSLOduration=3.009912737 podStartE2EDuration="3.490621597s" totalImagesPullingTime="480.70886ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:34:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:00.487903375 +0000 UTC m=+1732.228894452" watchObservedRunningTime="2026-10-30 19:35:00.490621597 +0000 UTC m=+1732.231612675"
Oct 30 19:35:00.517652 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.517619    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"
Oct 30 19:35:00.813655 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:00.813627    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"]
Oct 30 19:35:00.816445 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:35:00.816312    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95bd6255_4eb5_40ba_a80b_9e387b4c5c3a.slice/crio-b3ea516bc0cd5e80b62d5f50cde362c9d1b2e3b44352c93cde3b7b35155872cf WatchSource:0}: Error finding container b3ea516bc0cd5e80b62d5f50cde362c9d1b2e3b44352c93cde3b7b35155872cf: Status 404 returned error can't find the container with id b3ea516bc0cd5e80b62d5f50cde362c9d1b2e3b44352c93cde3b7b35155872cf
Oct 30 19:35:01.475079 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:01.475039    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs" event={"ID":"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a","Type":"ContainerStarted","Data":"af54ef1f63be9305f9f565ee3a084a6dd652fc3440e33d72eb5a38f6469e2ed2"}
Oct 30 19:35:01.475314 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:01.475106    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs" event={"ID":"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a","Type":"ContainerStarted","Data":"b3ea516bc0cd5e80b62d5f50cde362c9d1b2e3b44352c93cde3b7b35155872cf"}
Oct 30 19:35:01.475314 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:01.475136    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"
Oct 30 19:35:01.497985 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:01.497928    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs" podStartSLOduration=1.015622711 podStartE2EDuration="1.497914244s" totalImagesPullingTime="482.291533ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:01.49577423 +0000 UTC m=+1733.236765310" watchObservedRunningTime="2026-10-30 19:35:01.497914244 +0000 UTC m=+1733.238905322"
Oct 30 19:35:02.488264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:02.488231    4524 generic.go:356] "Generic (PLEG): container finished" podID="0662c903-51a8-4e05-a90d-f3baf5488511" containerID="69f864fbf3d4bfecde11a3523fa3d60c50e38888bc29cf06b2c101c4a9cb1152" exitCode=0
Oct 30 19:35:02.488707 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:02.488302    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29" event={"ID":"0662c903-51a8-4e05-a90d-f3baf5488511","Type":"ContainerDied","Data":"69f864fbf3d4bfecde11a3523fa3d60c50e38888bc29cf06b2c101c4a9cb1152"}
Oct 30 19:35:02.661325 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:02.661300    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29"
Oct 30 19:35:02.708942 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:02.708845    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0662c903-51a8-4e05-a90d-f3baf5488511-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0662c903-51a8-4e05-a90d-f3baf5488511-plugins-config-volume\") pod \"0662c903-51a8-4e05-a90d-f3baf5488511\" (UID: \"0662c903-51a8-4e05-a90d-f3baf5488511\") "
Oct 30 19:35:02.709200 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:02.709159    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0662c903-51a8-4e05-a90d-f3baf5488511-kube-api-access-dmhcs\" (UniqueName: \"kubernetes.io/projected/0662c903-51a8-4e05-a90d-f3baf5488511-kube-api-access-dmhcs\") pod \"0662c903-51a8-4e05-a90d-f3baf5488511\" (UID: \"0662c903-51a8-4e05-a90d-f3baf5488511\") "
Oct 30 19:35:02.709398 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:02.709369    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0662c903-51a8-4e05-a90d-f3baf5488511-plugins-config-volume" pod "0662c903-51a8-4e05-a90d-f3baf5488511" (UID: "0662c903-51a8-4e05-a90d-f3baf5488511"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:35:02.709977 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:02.709854    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0662c903-51a8-4e05-a90d-f3baf5488511-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:02.711956 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:02.711927    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0662c903-51a8-4e05-a90d-f3baf5488511-kube-api-access-dmhcs" pod "0662c903-51a8-4e05-a90d-f3baf5488511" (UID: "0662c903-51a8-4e05-a90d-f3baf5488511"). InnerVolumeSpecName "kube-api-access-dmhcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:02.812956 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:02.812913    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmhcs\" (UniqueName: \"kubernetes.io/projected/0662c903-51a8-4e05-a90d-f3baf5488511-kube-api-access-dmhcs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:03.497724 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:03.497695    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29"
Oct 30 19:35:03.498210 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:03.497734    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29" event={"ID":"0662c903-51a8-4e05-a90d-f3baf5488511","Type":"ContainerDied","Data":"f9aa17f285b9758014e59ad27c1288a7311e866dd612f81cac39be10db823ae7"}
Oct 30 19:35:03.498210 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:03.497794    4524 scope.go:122] "RemoveContainer" containerID="69f864fbf3d4bfecde11a3523fa3d60c50e38888bc29cf06b2c101c4a9cb1152"
Oct 30 19:35:03.529156 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:03.529129    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29"]
Oct 30 19:35:03.551964 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:03.551926    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-afdf16-7db74769f4-nht29"]
Oct 30 19:35:04.395520 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:04.395487    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0662c903-51a8-4e05-a90d-f3baf5488511" path="/var/lib/kubelet/pods/0662c903-51a8-4e05-a90d-f3baf5488511/volumes"
Oct 30 19:35:06.483230 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:06.483192    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"
Oct 30 19:35:06.754000 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:06.753905    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"]
Oct 30 19:35:06.754200 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:06.754157    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg" podUID="b536022f-7c4e-436f-a804-1c7bf324e0f0" containerName="maas-api" containerID="cri-o://2c608f06fc79a7a13bda907dc5f2dd3b091be1b62404c5b820e3e9a1552c55d3" gracePeriod=30
Oct 30 19:35:07.493910 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.493889    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"
Oct 30 19:35:07.520187 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.520153    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"
Oct 30 19:35:07.616241 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.613378    4524 generic.go:356] "Generic (PLEG): container finished" podID="b536022f-7c4e-436f-a804-1c7bf324e0f0" containerID="2c608f06fc79a7a13bda907dc5f2dd3b091be1b62404c5b820e3e9a1552c55d3" exitCode=0
Oct 30 19:35:07.616241 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.613451    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg" event={"ID":"b536022f-7c4e-436f-a804-1c7bf324e0f0","Type":"ContainerDied","Data":"2c608f06fc79a7a13bda907dc5f2dd3b091be1b62404c5b820e3e9a1552c55d3"}
Oct 30 19:35:07.616241 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.613548    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg" event={"ID":"b536022f-7c4e-436f-a804-1c7bf324e0f0","Type":"ContainerDied","Data":"fb31475595091d8d6f698882abe260273c6e56bc7739be7976bb24cbf457641b"}
Oct 30 19:35:07.616241 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.613587    4524 scope.go:122] "RemoveContainer" containerID="2c608f06fc79a7a13bda907dc5f2dd3b091be1b62404c5b820e3e9a1552c55d3"
Oct 30 19:35:07.616241 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.613819    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"
Oct 30 19:35:07.637208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.631656    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b536022f-7c4e-436f-a804-1c7bf324e0f0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b536022f-7c4e-436f-a804-1c7bf324e0f0-openshift-service-ca\") pod \"b536022f-7c4e-436f-a804-1c7bf324e0f0\" (UID: \"b536022f-7c4e-436f-a804-1c7bf324e0f0\") "
Oct 30 19:35:07.637208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.631784    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b536022f-7c4e-436f-a804-1c7bf324e0f0-kube-api-access-z2cpr\" (UniqueName: \"kubernetes.io/projected/b536022f-7c4e-436f-a804-1c7bf324e0f0-kube-api-access-z2cpr\") pod \"b536022f-7c4e-436f-a804-1c7bf324e0f0\" (UID: \"b536022f-7c4e-436f-a804-1c7bf324e0f0\") "
Oct 30 19:35:07.637208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.631872    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b536022f-7c4e-436f-a804-1c7bf324e0f0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b536022f-7c4e-436f-a804-1c7bf324e0f0-maas-api-tls\") pod \"b536022f-7c4e-436f-a804-1c7bf324e0f0\" (UID: \"b536022f-7c4e-436f-a804-1c7bf324e0f0\") "
Oct 30 19:35:07.637208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.631911    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b536022f-7c4e-436f-a804-1c7bf324e0f0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b536022f-7c4e-436f-a804-1c7bf324e0f0-metrics-certs\") pod \"b536022f-7c4e-436f-a804-1c7bf324e0f0\" (UID: \"b536022f-7c4e-436f-a804-1c7bf324e0f0\") "
Oct 30 19:35:07.637208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.633237    4524 scope.go:122] "RemoveContainer" containerID="2c608f06fc79a7a13bda907dc5f2dd3b091be1b62404c5b820e3e9a1552c55d3"
Oct 30 19:35:07.641230 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.641151    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b536022f-7c4e-436f-a804-1c7bf324e0f0-openshift-service-ca" pod "b536022f-7c4e-436f-a804-1c7bf324e0f0" (UID: "b536022f-7c4e-436f-a804-1c7bf324e0f0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:35:07.641366 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:35:07.641333    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c608f06fc79a7a13bda907dc5f2dd3b091be1b62404c5b820e3e9a1552c55d3\": container with ID starting with 2c608f06fc79a7a13bda907dc5f2dd3b091be1b62404c5b820e3e9a1552c55d3 not found: ID does not exist" containerID="2c608f06fc79a7a13bda907dc5f2dd3b091be1b62404c5b820e3e9a1552c55d3"
Oct 30 19:35:07.641437 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.641381    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c608f06fc79a7a13bda907dc5f2dd3b091be1b62404c5b820e3e9a1552c55d3"} err="failed to get container status \"2c608f06fc79a7a13bda907dc5f2dd3b091be1b62404c5b820e3e9a1552c55d3\": rpc error: code = NotFound desc = could not find container \"2c608f06fc79a7a13bda907dc5f2dd3b091be1b62404c5b820e3e9a1552c55d3\": container with ID starting with 2c608f06fc79a7a13bda907dc5f2dd3b091be1b62404c5b820e3e9a1552c55d3 not found: ID does not exist"
Oct 30 19:35:07.642273 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.642245    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b536022f-7c4e-436f-a804-1c7bf324e0f0-maas-api-tls" pod "b536022f-7c4e-436f-a804-1c7bf324e0f0" (UID: "b536022f-7c4e-436f-a804-1c7bf324e0f0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:35:07.647792 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.647755    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b536022f-7c4e-436f-a804-1c7bf324e0f0-metrics-certs" pod "b536022f-7c4e-436f-a804-1c7bf324e0f0" (UID: "b536022f-7c4e-436f-a804-1c7bf324e0f0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:35:07.647893 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.647827    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b536022f-7c4e-436f-a804-1c7bf324e0f0-kube-api-access-z2cpr" pod "b536022f-7c4e-436f-a804-1c7bf324e0f0" (UID: "b536022f-7c4e-436f-a804-1c7bf324e0f0"). InnerVolumeSpecName "kube-api-access-z2cpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:07.737569 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.737533    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2cpr\" (UniqueName: \"kubernetes.io/projected/b536022f-7c4e-436f-a804-1c7bf324e0f0-kube-api-access-z2cpr\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:07.737569 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.737565    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b536022f-7c4e-436f-a804-1c7bf324e0f0-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:07.737569 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.737575    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b536022f-7c4e-436f-a804-1c7bf324e0f0-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:07.737802 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.737583    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b536022f-7c4e-436f-a804-1c7bf324e0f0-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:07.944246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.944208    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"]
Oct 30 19:35:07.949425 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:07.949399    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d52364-648b4f5f6c-6nfcg"]
Oct 30 19:35:08.401835 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:08.401802    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b536022f-7c4e-436f-a804-1c7bf324e0f0" path="/var/lib/kubelet/pods/b536022f-7c4e-436f-a804-1c7bf324e0f0/volumes"
Oct 30 19:35:10.801308 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:10.801273    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"]
Oct 30 19:35:10.801802 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:10.801530    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc" podUID="486a1648-1812-4504-83db-1c53357bd89f" containerName="maas-api" containerID="cri-o://3c69465f581cfd7e45f7ebe7b50bb484d082396076eb169070f352a97682f7c5" gracePeriod=30
Oct 30 19:35:11.734393 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:11.734357    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc" event={"ID":"486a1648-1812-4504-83db-1c53357bd89f","Type":"ContainerDied","Data":"3c69465f581cfd7e45f7ebe7b50bb484d082396076eb169070f352a97682f7c5"}
Oct 30 19:35:11.734793 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:11.734287    4524 generic.go:356] "Generic (PLEG): container finished" podID="486a1648-1812-4504-83db-1c53357bd89f" containerID="3c69465f581cfd7e45f7ebe7b50bb484d082396076eb169070f352a97682f7c5" exitCode=0
Oct 30 19:35:11.750967 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:11.750808    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"
Oct 30 19:35:11.853949 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:11.853910    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/486a1648-1812-4504-83db-1c53357bd89f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/486a1648-1812-4504-83db-1c53357bd89f-metrics-certs\") pod \"486a1648-1812-4504-83db-1c53357bd89f\" (UID: \"486a1648-1812-4504-83db-1c53357bd89f\") "
Oct 30 19:35:11.854514 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:11.854046    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/486a1648-1812-4504-83db-1c53357bd89f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/486a1648-1812-4504-83db-1c53357bd89f-openshift-service-ca\") pod \"486a1648-1812-4504-83db-1c53357bd89f\" (UID: \"486a1648-1812-4504-83db-1c53357bd89f\") "
Oct 30 19:35:11.854514 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:11.854110    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/486a1648-1812-4504-83db-1c53357bd89f-kube-api-access-tml64\" (UniqueName: \"kubernetes.io/projected/486a1648-1812-4504-83db-1c53357bd89f-kube-api-access-tml64\") pod \"486a1648-1812-4504-83db-1c53357bd89f\" (UID: \"486a1648-1812-4504-83db-1c53357bd89f\") "
Oct 30 19:35:11.854514 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:11.854158    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/486a1648-1812-4504-83db-1c53357bd89f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/486a1648-1812-4504-83db-1c53357bd89f-maas-api-tls\") pod \"486a1648-1812-4504-83db-1c53357bd89f\" (UID: \"486a1648-1812-4504-83db-1c53357bd89f\") "
Oct 30 19:35:11.855147 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:11.855116    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/486a1648-1812-4504-83db-1c53357bd89f-openshift-service-ca" pod "486a1648-1812-4504-83db-1c53357bd89f" (UID: "486a1648-1812-4504-83db-1c53357bd89f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:35:11.856742 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:11.856703    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/486a1648-1812-4504-83db-1c53357bd89f-metrics-certs" pod "486a1648-1812-4504-83db-1c53357bd89f" (UID: "486a1648-1812-4504-83db-1c53357bd89f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:35:11.857711 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:11.857689    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/486a1648-1812-4504-83db-1c53357bd89f-kube-api-access-tml64" pod "486a1648-1812-4504-83db-1c53357bd89f" (UID: "486a1648-1812-4504-83db-1c53357bd89f"). InnerVolumeSpecName "kube-api-access-tml64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:11.857869 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:11.857853    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/486a1648-1812-4504-83db-1c53357bd89f-maas-api-tls" pod "486a1648-1812-4504-83db-1c53357bd89f" (UID: "486a1648-1812-4504-83db-1c53357bd89f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:35:11.956319 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:11.956280    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/486a1648-1812-4504-83db-1c53357bd89f-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:11.956319 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:11.956318    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/486a1648-1812-4504-83db-1c53357bd89f-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:11.956536 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:11.956332    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tml64\" (UniqueName: \"kubernetes.io/projected/486a1648-1812-4504-83db-1c53357bd89f-kube-api-access-tml64\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:11.956536 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:11.956346    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/486a1648-1812-4504-83db-1c53357bd89f-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:12.746587 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:12.746497    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"
Oct 30 19:35:12.746587 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:12.746532    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc" event={"ID":"486a1648-1812-4504-83db-1c53357bd89f","Type":"ContainerDied","Data":"f51e89d09e34001e86a0eac4d8fe75dec453ce19dee01e45c9b0c20c1325ed08"}
Oct 30 19:35:12.746825 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:12.746598    4524 scope.go:122] "RemoveContainer" containerID="3c69465f581cfd7e45f7ebe7b50bb484d082396076eb169070f352a97682f7c5"
Oct 30 19:35:12.767410 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:12.767382    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"]
Oct 30 19:35:12.771653 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:12.771626    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-02feaa-58ff9dd844-gdpxc"]
Oct 30 19:35:13.053425 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:13.053328    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4"]
Oct 30 19:35:13.062101 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:13.062061    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-2596eef9-rkpz4"]
Oct 30 19:35:14.392371 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:14.392337    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="486a1648-1812-4504-83db-1c53357bd89f" path="/var/lib/kubelet/pods/486a1648-1812-4504-83db-1c53357bd89f/volumes"
Oct 30 19:35:14.392828 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:14.392813    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc886570-4bd4-4f6d-8d49-2bcb11921b94" path="/var/lib/kubelet/pods/cc886570-4bd4-4f6d-8d49-2bcb11921b94/volumes"
Oct 30 19:35:15.665136 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:35:15.665094    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:35:21.419523 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:21.419441    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"]
Oct 30 19:35:21.420036 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:21.419696    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb" podUID="849295c8-38ec-4630-95be-60a6200e4a20" containerName="istio-proxy" containerID="cri-o://2b23122467660529993982cfcf2c7902b918d4514263538d764e1dc44cffc662" gracePeriod=30
Oct 30 19:35:21.501412 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:21.501381    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"]
Oct 30 19:35:21.501631 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:21.501606    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9" podUID="22c6a6f8-5d6b-4607-85b1-881a6a08f361" containerName="istio-proxy" containerID="cri-o://eeba74b9d84738467ad94bccb4b77b32665329861b0a6d3ec9001f84148eae22" gracePeriod=30
Oct 30 19:35:23.378667 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.378630    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"]
Oct 30 19:35:23.386991 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.386961    4524 cpu_manager.go:414] "containerMap: removing container" podUID="0662c903-51a8-4e05-a90d-f3baf5488511" containerName="payload-processing"
Oct 30 19:35:23.386991 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.386990    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0662c903-51a8-4e05-a90d-f3baf5488511" containerName="payload-processing"
Oct 30 19:35:23.387452 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.387019    4524 cpu_manager.go:414] "containerMap: removing container" podUID="b536022f-7c4e-436f-a804-1c7bf324e0f0" containerName="maas-api"
Oct 30 19:35:23.387452 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.387028    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b536022f-7c4e-436f-a804-1c7bf324e0f0" containerName="maas-api"
Oct 30 19:35:23.387452 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.387083    4524 cpu_manager.go:414] "containerMap: removing container" podUID="486a1648-1812-4504-83db-1c53357bd89f" containerName="maas-api"
Oct 30 19:35:23.387452 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.387090    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="486a1648-1812-4504-83db-1c53357bd89f" containerName="maas-api"
Oct 30 19:35:23.387912 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.387893    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="486a1648-1812-4504-83db-1c53357bd89f" containerName="maas-api"
Oct 30 19:35:23.388000 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.387926    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="0662c903-51a8-4e05-a90d-f3baf5488511" containerName="payload-processing"
Oct 30 19:35:23.388000 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.387938    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="b536022f-7c4e-436f-a804-1c7bf324e0f0" containerName="maas-api"
Oct 30 19:35:23.394432 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.394400    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"
Oct 30 19:35:23.406025 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.403786    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-ddb6ea-serving-cert\""
Oct 30 19:35:23.413231 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.413052    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"]
Oct 30 19:35:23.524873 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.524223    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9xf2\" (UniqueName: \"kubernetes.io/projected/6fb90c94-7e88-4cd8-96c1-f779013b1694-kube-api-access-j9xf2\") pod \"maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs\" (UID: \"6fb90c94-7e88-4cd8-96c1-f779013b1694\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"
Oct 30 19:35:23.524873 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.524402    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb90c94-7e88-4cd8-96c1-f779013b1694-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs\" (UID: \"6fb90c94-7e88-4cd8-96c1-f779013b1694\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"
Oct 30 19:35:23.524873 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.524536    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fb90c94-7e88-4cd8-96c1-f779013b1694-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs\" (UID: \"6fb90c94-7e88-4cd8-96c1-f779013b1694\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"
Oct 30 19:35:23.526477 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.526137    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb90c94-7e88-4cd8-96c1-f779013b1694-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs\" (UID: \"6fb90c94-7e88-4cd8-96c1-f779013b1694\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"
Oct 30 19:35:23.630331 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.630094    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb90c94-7e88-4cd8-96c1-f779013b1694-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs\" (UID: \"6fb90c94-7e88-4cd8-96c1-f779013b1694\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"
Oct 30 19:35:23.631877 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.631852    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb90c94-7e88-4cd8-96c1-f779013b1694-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs\" (UID: \"6fb90c94-7e88-4cd8-96c1-f779013b1694\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"
Oct 30 19:35:23.632202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.632186    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9xf2\" (UniqueName: \"kubernetes.io/projected/6fb90c94-7e88-4cd8-96c1-f779013b1694-kube-api-access-j9xf2\") pod \"maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs\" (UID: \"6fb90c94-7e88-4cd8-96c1-f779013b1694\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"
Oct 30 19:35:23.632404 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.632386    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb90c94-7e88-4cd8-96c1-f779013b1694-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs\" (UID: \"6fb90c94-7e88-4cd8-96c1-f779013b1694\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"
Oct 30 19:35:23.632610 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.632595    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fb90c94-7e88-4cd8-96c1-f779013b1694-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs\" (UID: \"6fb90c94-7e88-4cd8-96c1-f779013b1694\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"
Oct 30 19:35:23.654255 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.645863    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fb90c94-7e88-4cd8-96c1-f779013b1694-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs\" (UID: \"6fb90c94-7e88-4cd8-96c1-f779013b1694\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"
Oct 30 19:35:23.668499 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.666952    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9xf2\" (UniqueName: \"kubernetes.io/projected/6fb90c94-7e88-4cd8-96c1-f779013b1694-kube-api-access-j9xf2\") pod \"maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs\" (UID: \"6fb90c94-7e88-4cd8-96c1-f779013b1694\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"
Oct 30 19:35:23.668499 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.667471    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb90c94-7e88-4cd8-96c1-f779013b1694-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs\" (UID: \"6fb90c94-7e88-4cd8-96c1-f779013b1694\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"
Oct 30 19:35:23.726229 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:23.726155    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"
Oct 30 19:35:24.064385 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:35:24.064353    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fb90c94_7e88_4cd8_96c1_f779013b1694.slice/crio-b1b751a8c9ce625bbe0b55a21d411c2428f840160d00a51fb48f1e6ed2740c39 WatchSource:0}: Error finding container b1b751a8c9ce625bbe0b55a21d411c2428f840160d00a51fb48f1e6ed2740c39: Status 404 returned error can't find the container with id b1b751a8c9ce625bbe0b55a21d411c2428f840160d00a51fb48f1e6ed2740c39
Oct 30 19:35:24.064765 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:24.064739    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"]
Oct 30 19:35:24.929549 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:24.928874    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs" event={"ID":"6fb90c94-7e88-4cd8-96c1-f779013b1694","Type":"ContainerStarted","Data":"67f952639a0536feff2cb15669432494070cf290a293463f09a54fc9d5c5ed32"}
Oct 30 19:35:24.929549 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:24.928999    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs" event={"ID":"6fb90c94-7e88-4cd8-96c1-f779013b1694","Type":"ContainerStarted","Data":"b1b751a8c9ce625bbe0b55a21d411c2428f840160d00a51fb48f1e6ed2740c39"}
Oct 30 19:35:24.929549 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:24.929065    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"
Oct 30 19:35:24.949453 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:24.949397    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs" podStartSLOduration=1.459910957 podStartE2EDuration="1.949381965s" totalImagesPullingTime="489.471008ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:24.947839185 +0000 UTC m=+1756.688830266" watchObservedRunningTime="2026-10-30 19:35:24.949381965 +0000 UTC m=+1756.690373043"
Oct 30 19:35:26.971677 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:26.971646    4524 generic.go:356] "Generic (PLEG): container finished" podID="849295c8-38ec-4630-95be-60a6200e4a20" containerID="2b23122467660529993982cfcf2c7902b918d4514263538d764e1dc44cffc662" exitCode=0
Oct 30 19:35:26.972142 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:26.971718    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb" event={"ID":"849295c8-38ec-4630-95be-60a6200e4a20","Type":"ContainerDied","Data":"2b23122467660529993982cfcf2c7902b918d4514263538d764e1dc44cffc662"}
Oct 30 19:35:26.973681 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:26.973647    4524 generic.go:356] "Generic (PLEG): container finished" podID="22c6a6f8-5d6b-4607-85b1-881a6a08f361" containerID="eeba74b9d84738467ad94bccb4b77b32665329861b0a6d3ec9001f84148eae22" exitCode=0
Oct 30 19:35:26.973822 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:26.973734    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9" event={"ID":"22c6a6f8-5d6b-4607-85b1-881a6a08f361","Type":"ContainerDied","Data":"eeba74b9d84738467ad94bccb4b77b32665329861b0a6d3ec9001f84148eae22"}
Oct 30 19:35:27.022689 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.022666    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:35:27.050199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.046483    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:35:27.177207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.169698    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-credential-socket\") pod \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") "
Oct 30 19:35:27.177207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.169772    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22c6a6f8-5d6b-4607-85b1-881a6a08f361-kube-api-access-r56r9\" (UniqueName: \"kubernetes.io/projected/22c6a6f8-5d6b-4607-85b1-881a6a08f361-kube-api-access-r56r9\") pod \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") "
Oct 30 19:35:27.177207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.169824    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-envoy\") pod \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") "
Oct 30 19:35:27.177207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.169889    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/849295c8-38ec-4630-95be-60a6200e4a20-kube-api-access-hx7vz\" (UniqueName: \"kubernetes.io/projected/849295c8-38ec-4630-95be-60a6200e4a20-kube-api-access-hx7vz\") pod \"849295c8-38ec-4630-95be-60a6200e4a20\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") "
Oct 30 19:35:27.177207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.169921    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-podinfo\") pod \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") "
Oct 30 19:35:27.177207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.169961    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-istio-data\") pod \"849295c8-38ec-4630-95be-60a6200e4a20\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") "
Oct 30 19:35:27.177207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.169996    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-workload-certs\") pod \"849295c8-38ec-4630-95be-60a6200e4a20\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") "
Oct 30 19:35:27.177207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.170034    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/849295c8-38ec-4630-95be-60a6200e4a20-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/849295c8-38ec-4630-95be-60a6200e4a20-istiod-ca-cert\") pod \"849295c8-38ec-4630-95be-60a6200e4a20\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") "
Oct 30 19:35:27.177207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.170065    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istiod-ca-cert\") pod \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") "
Oct 30 19:35:27.177207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.170108    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-workload-socket\") pod \"849295c8-38ec-4630-95be-60a6200e4a20\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") "
Oct 30 19:35:27.177207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.170196    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-istio-envoy\") pod \"849295c8-38ec-4630-95be-60a6200e4a20\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") "
Oct 30 19:35:27.177207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.170233    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-workload-certs\") pod \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") "
Oct 30 19:35:27.177207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.170403    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/849295c8-38ec-4630-95be-60a6200e4a20-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/849295c8-38ec-4630-95be-60a6200e4a20-istio-podinfo\") pod \"849295c8-38ec-4630-95be-60a6200e4a20\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") "
Oct 30 19:35:27.177207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.170451    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-workload-socket\") pod \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") "
Oct 30 19:35:27.177207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.170525    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/849295c8-38ec-4630-95be-60a6200e4a20-istio-token\" (UniqueName: \"kubernetes.io/projected/849295c8-38ec-4630-95be-60a6200e4a20-istio-token\") pod \"849295c8-38ec-4630-95be-60a6200e4a20\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") "
Oct 30 19:35:27.177207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.170587    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-token\" (UniqueName: \"kubernetes.io/projected/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-token\") pod \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") "
Oct 30 19:35:27.177207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.170642    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-data\") pod \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\" (UID: \"22c6a6f8-5d6b-4607-85b1-881a6a08f361\") "
Oct 30 19:35:27.177207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.170698    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-credential-socket\") pod \"849295c8-38ec-4630-95be-60a6200e4a20\" (UID: \"849295c8-38ec-4630-95be-60a6200e4a20\") "
Oct 30 19:35:27.177207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.173541    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istiod-ca-cert" pod "22c6a6f8-5d6b-4607-85b1-881a6a08f361" (UID: "22c6a6f8-5d6b-4607-85b1-881a6a08f361"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:35:27.177207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.173818    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-credential-socket" pod "22c6a6f8-5d6b-4607-85b1-881a6a08f361" (UID: "22c6a6f8-5d6b-4607-85b1-881a6a08f361"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:27.178399 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.177250    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-credential-socket" pod "849295c8-38ec-4630-95be-60a6200e4a20" (UID: "849295c8-38ec-4630-95be-60a6200e4a20"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:27.178399 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.177486    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-workload-socket" pod "22c6a6f8-5d6b-4607-85b1-881a6a08f361" (UID: "22c6a6f8-5d6b-4607-85b1-881a6a08f361"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:27.184077 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.183569    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-workload-certs" pod "849295c8-38ec-4630-95be-60a6200e4a20" (UID: "849295c8-38ec-4630-95be-60a6200e4a20"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:27.184077 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.183786    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-istio-data" pod "849295c8-38ec-4630-95be-60a6200e4a20" (UID: "849295c8-38ec-4630-95be-60a6200e4a20"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:27.187076 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.186890    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/849295c8-38ec-4630-95be-60a6200e4a20-istio-podinfo" pod "849295c8-38ec-4630-95be-60a6200e4a20" (UID: "849295c8-38ec-4630-95be-60a6200e4a20"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:35:27.187562 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.187377    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-workload-certs" pod "22c6a6f8-5d6b-4607-85b1-881a6a08f361" (UID: "22c6a6f8-5d6b-4607-85b1-881a6a08f361"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:27.188125 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.187880    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/849295c8-38ec-4630-95be-60a6200e4a20-istiod-ca-cert" pod "849295c8-38ec-4630-95be-60a6200e4a20" (UID: "849295c8-38ec-4630-95be-60a6200e4a20"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:35:27.188744 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.188479    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/849295c8-38ec-4630-95be-60a6200e4a20-istio-token" pod "849295c8-38ec-4630-95be-60a6200e4a20" (UID: "849295c8-38ec-4630-95be-60a6200e4a20"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:27.189126 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.188872    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-data" pod "22c6a6f8-5d6b-4607-85b1-881a6a08f361" (UID: "22c6a6f8-5d6b-4607-85b1-881a6a08f361"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:27.191403 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.191371    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/849295c8-38ec-4630-95be-60a6200e4a20-kube-api-access-hx7vz" pod "849295c8-38ec-4630-95be-60a6200e4a20" (UID: "849295c8-38ec-4630-95be-60a6200e4a20"). InnerVolumeSpecName "kube-api-access-hx7vz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:27.194822 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.192273    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-envoy" pod "22c6a6f8-5d6b-4607-85b1-881a6a08f361" (UID: "22c6a6f8-5d6b-4607-85b1-881a6a08f361"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:27.198811 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.198764    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-istio-envoy" pod "849295c8-38ec-4630-95be-60a6200e4a20" (UID: "849295c8-38ec-4630-95be-60a6200e4a20"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:27.199090 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.199051    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22c6a6f8-5d6b-4607-85b1-881a6a08f361-kube-api-access-r56r9" pod "22c6a6f8-5d6b-4607-85b1-881a6a08f361" (UID: "22c6a6f8-5d6b-4607-85b1-881a6a08f361"). InnerVolumeSpecName "kube-api-access-r56r9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:27.207220 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.206238    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-workload-socket" pod "849295c8-38ec-4630-95be-60a6200e4a20" (UID: "849295c8-38ec-4630-95be-60a6200e4a20"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:27.291727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.274556    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/849295c8-38ec-4630-95be-60a6200e4a20-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:27.291727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.274590    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:27.291727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.274607    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:27.291727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.274626    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:27.291727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.274643    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r56r9\" (UniqueName: \"kubernetes.io/projected/22c6a6f8-5d6b-4607-85b1-881a6a08f361-kube-api-access-r56r9\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:27.291727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.274658    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:27.291727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.274671    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hx7vz\" (UniqueName: \"kubernetes.io/projected/849295c8-38ec-4630-95be-60a6200e4a20-kube-api-access-hx7vz\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:27.291727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.274685    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:27.291727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.274699    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:27.291727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.274713    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/849295c8-38ec-4630-95be-60a6200e4a20-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:27.291727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.274727    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:27.291727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.274738    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:27.291727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.274753    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/849295c8-38ec-4630-95be-60a6200e4a20-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:27.291727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.274767    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:27.291727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.274780    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/849295c8-38ec-4630-95be-60a6200e4a20-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:27.291727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.274792    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22c6a6f8-5d6b-4607-85b1-881a6a08f361-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:27.291727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.277456    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-podinfo" pod "22c6a6f8-5d6b-4607-85b1-881a6a08f361" (UID: "22c6a6f8-5d6b-4607-85b1-881a6a08f361"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:35:27.291727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.277626    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-token" pod "22c6a6f8-5d6b-4607-85b1-881a6a08f361" (UID: "22c6a6f8-5d6b-4607-85b1-881a6a08f361"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:27.393192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.390546    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:27.393192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.390585    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22c6a6f8-5d6b-4607-85b1-881a6a08f361-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:27.984597 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.984565    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"
Oct 30 19:35:27.984597 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.984588    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb" event={"ID":"849295c8-38ec-4630-95be-60a6200e4a20","Type":"ContainerDied","Data":"d77e85499014fe0c25578b3a8baa2a03e74502afd15f87fa5ced101313982942"}
Oct 30 19:35:27.985126 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.984657    4524 scope.go:122] "RemoveContainer" containerID="2b23122467660529993982cfcf2c7902b918d4514263538d764e1dc44cffc662"
Oct 30 19:35:27.986655 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.986631    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9" event={"ID":"22c6a6f8-5d6b-4607-85b1-881a6a08f361","Type":"ContainerDied","Data":"5b1693faebe168c4562aa2471de3d387c437e737d39b105ac29b6bd483c6549d"}
Oct 30 19:35:27.986781 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:27.986721    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"
Oct 30 19:35:28.000101 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:28.000004    4524 scope.go:122] "RemoveContainer" containerID="eeba74b9d84738467ad94bccb4b77b32665329861b0a6d3ec9001f84148eae22"
Oct 30 19:35:28.019374 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:28.019347    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"]
Oct 30 19:35:28.023927 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:28.023899    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-lwnw9"]
Oct 30 19:35:28.031850 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:28.031816    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"]
Oct 30 19:35:28.035789 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:28.035768    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-rvdfb"]
Oct 30 19:35:28.395207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:28.395162    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="22c6a6f8-5d6b-4607-85b1-881a6a08f361" path="/var/lib/kubelet/pods/22c6a6f8-5d6b-4607-85b1-881a6a08f361/volumes"
Oct 30 19:35:28.395835 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:28.395815    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="849295c8-38ec-4630-95be-60a6200e4a20" path="/var/lib/kubelet/pods/849295c8-38ec-4630-95be-60a6200e4a20/volumes"
Oct 30 19:35:30.966190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:30.965639    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"
Oct 30 19:35:34.488887 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:34.488842    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"]
Oct 30 19:35:34.489449 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:34.489098    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg" podUID="a1ae4189-16e5-4d0d-b64e-1e660176cb4f" containerName="maas-api" containerID="cri-o://d4c86e5f2cb0ba6afbab5f8deb47009532096e94579c580ac4e6683ed6376ddb" gracePeriod=30
Oct 30 19:35:35.061069 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:35.061039    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"
Oct 30 19:35:35.062747 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:35.062715    4524 generic.go:356] "Generic (PLEG): container finished" podID="a1ae4189-16e5-4d0d-b64e-1e660176cb4f" containerID="d4c86e5f2cb0ba6afbab5f8deb47009532096e94579c580ac4e6683ed6376ddb" exitCode=0
Oct 30 19:35:35.062867 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:35.062774    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg" event={"ID":"a1ae4189-16e5-4d0d-b64e-1e660176cb4f","Type":"ContainerDied","Data":"d4c86e5f2cb0ba6afbab5f8deb47009532096e94579c580ac4e6683ed6376ddb"}
Oct 30 19:35:35.062914 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:35.062881    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg" event={"ID":"a1ae4189-16e5-4d0d-b64e-1e660176cb4f","Type":"ContainerDied","Data":"6f192a41defdf02de68825bd765ab73073d73c35c1151d83d8c3640c15ad1b5f"}
Oct 30 19:35:35.062987 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:35.062933    4524 scope.go:122] "RemoveContainer" containerID="d4c86e5f2cb0ba6afbab5f8deb47009532096e94579c580ac4e6683ed6376ddb"
Oct 30 19:35:35.075393 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:35.075348    4524 scope.go:122] "RemoveContainer" containerID="d4c86e5f2cb0ba6afbab5f8deb47009532096e94579c580ac4e6683ed6376ddb"
Oct 30 19:35:35.075879 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:35:35.075846    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4c86e5f2cb0ba6afbab5f8deb47009532096e94579c580ac4e6683ed6376ddb\": container with ID starting with d4c86e5f2cb0ba6afbab5f8deb47009532096e94579c580ac4e6683ed6376ddb not found: ID does not exist" containerID="d4c86e5f2cb0ba6afbab5f8deb47009532096e94579c580ac4e6683ed6376ddb"
Oct 30 19:35:35.075964 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:35.075893    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4c86e5f2cb0ba6afbab5f8deb47009532096e94579c580ac4e6683ed6376ddb"} err="failed to get container status \"d4c86e5f2cb0ba6afbab5f8deb47009532096e94579c580ac4e6683ed6376ddb\": rpc error: code = NotFound desc = could not find container \"d4c86e5f2cb0ba6afbab5f8deb47009532096e94579c580ac4e6683ed6376ddb\": container with ID starting with d4c86e5f2cb0ba6afbab5f8deb47009532096e94579c580ac4e6683ed6376ddb not found: ID does not exist"
Oct 30 19:35:35.087015 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:35.086983    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-openshift-service-ca\") pod \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\" (UID: \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\") "
Oct 30 19:35:35.087191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:35.087041    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-kube-api-access-mfbkp\" (UniqueName: \"kubernetes.io/projected/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-kube-api-access-mfbkp\") pod \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\" (UID: \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\") "
Oct 30 19:35:35.087421 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:35.087369    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-metrics-certs\") pod \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\" (UID: \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\") "
Oct 30 19:35:35.087421 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:35.087402    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-openshift-service-ca" pod "a1ae4189-16e5-4d0d-b64e-1e660176cb4f" (UID: "a1ae4189-16e5-4d0d-b64e-1e660176cb4f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:35:35.087716 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:35.087421    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-maas-api-tls\") pod \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\" (UID: \"a1ae4189-16e5-4d0d-b64e-1e660176cb4f\") "
Oct 30 19:35:35.088345 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:35.088322    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:35.090325 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:35.090297    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-kube-api-access-mfbkp" pod "a1ae4189-16e5-4d0d-b64e-1e660176cb4f" (UID: "a1ae4189-16e5-4d0d-b64e-1e660176cb4f"). InnerVolumeSpecName "kube-api-access-mfbkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:35.090739 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:35.090711    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-metrics-certs" pod "a1ae4189-16e5-4d0d-b64e-1e660176cb4f" (UID: "a1ae4189-16e5-4d0d-b64e-1e660176cb4f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:35:35.090833 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:35.090735    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-maas-api-tls" pod "a1ae4189-16e5-4d0d-b64e-1e660176cb4f" (UID: "a1ae4189-16e5-4d0d-b64e-1e660176cb4f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:35:35.189573 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:35.189539    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:35.189573 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:35.189568    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:35.189771 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:35.189580    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfbkp\" (UniqueName: \"kubernetes.io/projected/a1ae4189-16e5-4d0d-b64e-1e660176cb4f-kube-api-access-mfbkp\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:36.122197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:36.113041    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"
Oct 30 19:35:36.149070 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:36.149034    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"]
Oct 30 19:35:36.156029 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:36.156002    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-c86696-6cf6fbc986-xksgg"]
Oct 30 19:35:36.396708 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:36.396618    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1ae4189-16e5-4d0d-b64e-1e660176cb4f" path="/var/lib/kubelet/pods/a1ae4189-16e5-4d0d-b64e-1e660176cb4f/volumes"
Oct 30 19:35:43.297690 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:43.297657    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"]
Oct 30 19:35:43.302535 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:43.302450    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm" podUID="968cf343-2e3b-4186-875d-2275e313a4b3" containerName="istio-proxy" containerID="cri-o://a378ec039dfab88492222047cf2b85b380407f2b2f0da3f70e915a4373a105ef" gracePeriod=30
Oct 30 19:35:45.628260 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:35:45.628219    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:35:46.882382 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:46.882340    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"]
Oct 30 19:35:46.883156 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:46.883116    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv" podUID="e24b4adc-d486-4aa9-a94d-d15cc1a30efb" containerName="maas-api" containerID="cri-o://8ebae01a749c180df287bcea399e1dc36b03e29439e7049ec86baf45ce6015bf" gracePeriod=30
Oct 30 19:35:47.001633 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:47.001596    4524 patch_prober.go:29] interesting pod/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:35:47.001930 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:47.001883    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm" podUID="968cf343-2e3b-4186-875d-2275e313a4b3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:35:47.404189 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:47.402728    4524 generic.go:356] "Generic (PLEG): container finished" podID="e24b4adc-d486-4aa9-a94d-d15cc1a30efb" containerID="8ebae01a749c180df287bcea399e1dc36b03e29439e7049ec86baf45ce6015bf" exitCode=0
Oct 30 19:35:47.404189 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:47.403034    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv" event={"ID":"e24b4adc-d486-4aa9-a94d-d15cc1a30efb","Type":"ContainerDied","Data":"8ebae01a749c180df287bcea399e1dc36b03e29439e7049ec86baf45ce6015bf"}
Oct 30 19:35:47.592192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:47.580375    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"
Oct 30 19:35:47.616193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:47.613633    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-openshift-service-ca\") pod \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\" (UID: \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\") "
Oct 30 19:35:47.616193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:47.613753    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-metrics-certs\") pod \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\" (UID: \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\") "
Oct 30 19:35:47.616193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:47.613880    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-maas-api-tls\") pod \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\" (UID: \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\") "
Oct 30 19:35:47.616193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:47.613939    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-kube-api-access-mhspx\" (UniqueName: \"kubernetes.io/projected/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-kube-api-access-mhspx\") pod \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\" (UID: \"e24b4adc-d486-4aa9-a94d-d15cc1a30efb\") "
Oct 30 19:35:47.627195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:47.619864    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-openshift-service-ca" pod "e24b4adc-d486-4aa9-a94d-d15cc1a30efb" (UID: "e24b4adc-d486-4aa9-a94d-d15cc1a30efb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:35:47.627195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:47.622971    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-kube-api-access-mhspx" pod "e24b4adc-d486-4aa9-a94d-d15cc1a30efb" (UID: "e24b4adc-d486-4aa9-a94d-d15cc1a30efb"). InnerVolumeSpecName "kube-api-access-mhspx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:47.627195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:47.624466    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-maas-api-tls" pod "e24b4adc-d486-4aa9-a94d-d15cc1a30efb" (UID: "e24b4adc-d486-4aa9-a94d-d15cc1a30efb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:35:47.627195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:47.626147    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-metrics-certs" pod "e24b4adc-d486-4aa9-a94d-d15cc1a30efb" (UID: "e24b4adc-d486-4aa9-a94d-d15cc1a30efb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:35:47.717962 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:47.717718    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhspx\" (UniqueName: \"kubernetes.io/projected/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-kube-api-access-mhspx\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:47.717962 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:47.717755    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:47.717962 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:47.717774    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:47.717962 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:47.717789    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e24b4adc-d486-4aa9-a94d-d15cc1a30efb-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:48.423363 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.423323    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv" event={"ID":"e24b4adc-d486-4aa9-a94d-d15cc1a30efb","Type":"ContainerDied","Data":"f2cf8b5bbcdb401b8a8bcfd28fa579150d230caac6b9f7feef85c4cd1955b71b"}
Oct 30 19:35:48.423831 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.423474    4524 scope.go:122] "RemoveContainer" containerID="8ebae01a749c180df287bcea399e1dc36b03e29439e7049ec86baf45ce6015bf"
Oct 30 19:35:48.423831 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.423718    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"
Oct 30 19:35:48.455525 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.455497    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"]
Oct 30 19:35:48.462759 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.462722    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"]
Oct 30 19:35:48.463876 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.463854    4524 cpu_manager.go:414] "containerMap: removing container" podUID="a1ae4189-16e5-4d0d-b64e-1e660176cb4f" containerName="maas-api"
Oct 30 19:35:48.463951 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.463878    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1ae4189-16e5-4d0d-b64e-1e660176cb4f" containerName="maas-api"
Oct 30 19:35:48.463951 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.463893    4524 cpu_manager.go:414] "containerMap: removing container" podUID="e24b4adc-d486-4aa9-a94d-d15cc1a30efb" containerName="maas-api"
Oct 30 19:35:48.463951 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.463901    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e24b4adc-d486-4aa9-a94d-d15cc1a30efb" containerName="maas-api"
Oct 30 19:35:48.463951 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.463936    4524 cpu_manager.go:414] "containerMap: removing container" podUID="849295c8-38ec-4630-95be-60a6200e4a20" containerName="istio-proxy"
Oct 30 19:35:48.463951 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.463943    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="849295c8-38ec-4630-95be-60a6200e4a20" containerName="istio-proxy"
Oct 30 19:35:48.464108 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.463979    4524 cpu_manager.go:414] "containerMap: removing container" podUID="22c6a6f8-5d6b-4607-85b1-881a6a08f361" containerName="istio-proxy"
Oct 30 19:35:48.464108 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.463986    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22c6a6f8-5d6b-4607-85b1-881a6a08f361" containerName="istio-proxy"
Oct 30 19:35:48.464183 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.464149    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1ae4189-16e5-4d0d-b64e-1e660176cb4f" containerName="maas-api"
Oct 30 19:35:48.464183 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.464162    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="e24b4adc-d486-4aa9-a94d-d15cc1a30efb" containerName="maas-api"
Oct 30 19:35:48.464251 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.464195    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="849295c8-38ec-4630-95be-60a6200e4a20" containerName="istio-proxy"
Oct 30 19:35:48.464251 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.464210    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="22c6a6f8-5d6b-4607-85b1-881a6a08f361" containerName="istio-proxy"
Oct 30 19:35:48.470303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.470278    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-69f3a9-6fc4755c5d-tzkzv"]
Oct 30 19:35:48.470409 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.470389    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"
Oct 30 19:35:48.487351 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.487326    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w6-0a6e66-serving-cert\""
Oct 30 19:35:48.498310 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.498272    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"]
Oct 30 19:35:48.551051 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.551019    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f88b8b61-5083-47c0-821d-92bf69941379-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt\" (UID: \"f88b8b61-5083-47c0-821d-92bf69941379\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"
Oct 30 19:35:48.551238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.551075    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9q9x\" (UniqueName: \"kubernetes.io/projected/f88b8b61-5083-47c0-821d-92bf69941379-kube-api-access-g9q9x\") pod \"maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt\" (UID: \"f88b8b61-5083-47c0-821d-92bf69941379\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"
Oct 30 19:35:48.551391 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.551366    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f88b8b61-5083-47c0-821d-92bf69941379-metrics-certs\") pod \"maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt\" (UID: \"f88b8b61-5083-47c0-821d-92bf69941379\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"
Oct 30 19:35:48.551455 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.551431    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f88b8b61-5083-47c0-821d-92bf69941379-maas-api-tls\") pod \"maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt\" (UID: \"f88b8b61-5083-47c0-821d-92bf69941379\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"
Oct 30 19:35:48.725607 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.725574    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9q9x\" (UniqueName: \"kubernetes.io/projected/f88b8b61-5083-47c0-821d-92bf69941379-kube-api-access-g9q9x\") pod \"maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt\" (UID: \"f88b8b61-5083-47c0-821d-92bf69941379\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"
Oct 30 19:35:48.726112 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.726075    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f88b8b61-5083-47c0-821d-92bf69941379-metrics-certs\") pod \"maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt\" (UID: \"f88b8b61-5083-47c0-821d-92bf69941379\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"
Oct 30 19:35:48.727055 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.727035    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f88b8b61-5083-47c0-821d-92bf69941379-maas-api-tls\") pod \"maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt\" (UID: \"f88b8b61-5083-47c0-821d-92bf69941379\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"
Oct 30 19:35:48.727607 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.727584    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f88b8b61-5083-47c0-821d-92bf69941379-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt\" (UID: \"f88b8b61-5083-47c0-821d-92bf69941379\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"
Oct 30 19:35:48.730244 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.730213    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f88b8b61-5083-47c0-821d-92bf69941379-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt\" (UID: \"f88b8b61-5083-47c0-821d-92bf69941379\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"
Oct 30 19:35:48.739467 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.739449    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f88b8b61-5083-47c0-821d-92bf69941379-maas-api-tls\") pod \"maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt\" (UID: \"f88b8b61-5083-47c0-821d-92bf69941379\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"
Oct 30 19:35:48.741377 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.741358    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f88b8b61-5083-47c0-821d-92bf69941379-metrics-certs\") pod \"maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt\" (UID: \"f88b8b61-5083-47c0-821d-92bf69941379\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"
Oct 30 19:35:48.767718 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.767692    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9q9x\" (UniqueName: \"kubernetes.io/projected/f88b8b61-5083-47c0-821d-92bf69941379-kube-api-access-g9q9x\") pod \"maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt\" (UID: \"f88b8b61-5083-47c0-821d-92bf69941379\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"
Oct 30 19:35:48.791182 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.788768    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"
Oct 30 19:35:48.797969 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.796701    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:35:48.932981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.931671    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-istio-data\") pod \"968cf343-2e3b-4186-875d-2275e313a4b3\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") "
Oct 30 19:35:48.932981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.931743    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/968cf343-2e3b-4186-875d-2275e313a4b3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/968cf343-2e3b-4186-875d-2275e313a4b3-istiod-ca-cert\") pod \"968cf343-2e3b-4186-875d-2275e313a4b3\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") "
Oct 30 19:35:48.932981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.931789    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/968cf343-2e3b-4186-875d-2275e313a4b3-kube-api-access-6hscs\" (UniqueName: \"kubernetes.io/projected/968cf343-2e3b-4186-875d-2275e313a4b3-kube-api-access-6hscs\") pod \"968cf343-2e3b-4186-875d-2275e313a4b3\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") "
Oct 30 19:35:48.932981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.931820    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-workload-certs\") pod \"968cf343-2e3b-4186-875d-2275e313a4b3\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") "
Oct 30 19:35:48.932981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.931912    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/968cf343-2e3b-4186-875d-2275e313a4b3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/968cf343-2e3b-4186-875d-2275e313a4b3-istio-podinfo\") pod \"968cf343-2e3b-4186-875d-2275e313a4b3\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") "
Oct 30 19:35:48.932981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.931931    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-credential-socket\") pod \"968cf343-2e3b-4186-875d-2275e313a4b3\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") "
Oct 30 19:35:48.932981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.931999    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-istio-envoy\") pod \"968cf343-2e3b-4186-875d-2275e313a4b3\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") "
Oct 30 19:35:48.932981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.932017    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/968cf343-2e3b-4186-875d-2275e313a4b3-istio-token\" (UniqueName: \"kubernetes.io/projected/968cf343-2e3b-4186-875d-2275e313a4b3-istio-token\") pod \"968cf343-2e3b-4186-875d-2275e313a4b3\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") "
Oct 30 19:35:48.932981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.932047    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-workload-socket\") pod \"968cf343-2e3b-4186-875d-2275e313a4b3\" (UID: \"968cf343-2e3b-4186-875d-2275e313a4b3\") "
Oct 30 19:35:48.932981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.932795    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-workload-certs" pod "968cf343-2e3b-4186-875d-2275e313a4b3" (UID: "968cf343-2e3b-4186-875d-2275e313a4b3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:48.933621 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.933009    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/968cf343-2e3b-4186-875d-2275e313a4b3-istiod-ca-cert" pod "968cf343-2e3b-4186-875d-2275e313a4b3" (UID: "968cf343-2e3b-4186-875d-2275e313a4b3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:35:48.933621 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.933133    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-istio-data" pod "968cf343-2e3b-4186-875d-2275e313a4b3" (UID: "968cf343-2e3b-4186-875d-2275e313a4b3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:48.933791 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.933755    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-credential-socket" pod "968cf343-2e3b-4186-875d-2275e313a4b3" (UID: "968cf343-2e3b-4186-875d-2275e313a4b3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:48.933869 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.933781    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-workload-socket" pod "968cf343-2e3b-4186-875d-2275e313a4b3" (UID: "968cf343-2e3b-4186-875d-2275e313a4b3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:48.936114 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.936090    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-istio-envoy" pod "968cf343-2e3b-4186-875d-2275e313a4b3" (UID: "968cf343-2e3b-4186-875d-2275e313a4b3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:35:48.936231 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.936187    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/968cf343-2e3b-4186-875d-2275e313a4b3-kube-api-access-6hscs" pod "968cf343-2e3b-4186-875d-2275e313a4b3" (UID: "968cf343-2e3b-4186-875d-2275e313a4b3"). InnerVolumeSpecName "kube-api-access-6hscs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:48.936303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.936278    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/968cf343-2e3b-4186-875d-2275e313a4b3-istio-token" pod "968cf343-2e3b-4186-875d-2275e313a4b3" (UID: "968cf343-2e3b-4186-875d-2275e313a4b3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:48.936347 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:48.936160    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/968cf343-2e3b-4186-875d-2275e313a4b3-istio-podinfo" pod "968cf343-2e3b-4186-875d-2275e313a4b3" (UID: "968cf343-2e3b-4186-875d-2275e313a4b3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:35:49.033870 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:49.033793    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hscs\" (UniqueName: \"kubernetes.io/projected/968cf343-2e3b-4186-875d-2275e313a4b3-kube-api-access-6hscs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:49.033870 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:49.033823    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:49.033870 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:49.033834    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/968cf343-2e3b-4186-875d-2275e313a4b3-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:49.033870 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:49.033846    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:49.033870 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:49.033860    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:49.033870 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:49.033868    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/968cf343-2e3b-4186-875d-2275e313a4b3-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:49.034202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:49.033878    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:49.034202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:49.033886    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/968cf343-2e3b-4186-875d-2275e313a4b3-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:49.034202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:49.033896    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/968cf343-2e3b-4186-875d-2275e313a4b3-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:49.290180 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:49.290002    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"]
Oct 30 19:35:49.435800 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:49.435756    4524 generic.go:356] "Generic (PLEG): container finished" podID="968cf343-2e3b-4186-875d-2275e313a4b3" containerID="a378ec039dfab88492222047cf2b85b380407f2b2f0da3f70e915a4373a105ef" exitCode=0
Oct 30 19:35:49.436303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:49.435813    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"
Oct 30 19:35:49.436303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:49.435845    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm" event={"ID":"968cf343-2e3b-4186-875d-2275e313a4b3","Type":"ContainerDied","Data":"a378ec039dfab88492222047cf2b85b380407f2b2f0da3f70e915a4373a105ef"}
Oct 30 19:35:49.436303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:49.435920    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm" event={"ID":"968cf343-2e3b-4186-875d-2275e313a4b3","Type":"ContainerDied","Data":"c4f804ef6cf5a97ea6dd2ba345d6684b1c3795b9d9d7d8b5e662767688b768b7"}
Oct 30 19:35:49.436303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:49.435969    4524 scope.go:122] "RemoveContainer" containerID="a378ec039dfab88492222047cf2b85b380407f2b2f0da3f70e915a4373a105ef"
Oct 30 19:35:49.440179 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:49.440126    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt" event={"ID":"f88b8b61-5083-47c0-821d-92bf69941379","Type":"ContainerStarted","Data":"80337aab057994f9d2b9e7b139c42b4b082cdfeb8fda49e08f9561039d2db24e"}
Oct 30 19:35:49.451213 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:49.451157    4524 scope.go:122] "RemoveContainer" containerID="a378ec039dfab88492222047cf2b85b380407f2b2f0da3f70e915a4373a105ef"
Oct 30 19:35:49.451490 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:35:49.451470    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a378ec039dfab88492222047cf2b85b380407f2b2f0da3f70e915a4373a105ef\": container with ID starting with a378ec039dfab88492222047cf2b85b380407f2b2f0da3f70e915a4373a105ef not found: ID does not exist" containerID="a378ec039dfab88492222047cf2b85b380407f2b2f0da3f70e915a4373a105ef"
Oct 30 19:35:49.451539 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:49.451497    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a378ec039dfab88492222047cf2b85b380407f2b2f0da3f70e915a4373a105ef"} err="failed to get container status \"a378ec039dfab88492222047cf2b85b380407f2b2f0da3f70e915a4373a105ef\": rpc error: code = NotFound desc = could not find container \"a378ec039dfab88492222047cf2b85b380407f2b2f0da3f70e915a4373a105ef\": container with ID starting with a378ec039dfab88492222047cf2b85b380407f2b2f0da3f70e915a4373a105ef not found: ID does not exist"
Oct 30 19:35:49.469829 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:49.469808    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"]
Oct 30 19:35:49.475107 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:49.475087    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-c86696-openshift-default-859496475c-9ttpm"]
Oct 30 19:35:50.398211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:50.397673    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="968cf343-2e3b-4186-875d-2275e313a4b3" path="/var/lib/kubelet/pods/968cf343-2e3b-4186-875d-2275e313a4b3/volumes"
Oct 30 19:35:50.400650 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:50.398760    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e24b4adc-d486-4aa9-a94d-d15cc1a30efb" path="/var/lib/kubelet/pods/e24b4adc-d486-4aa9-a94d-d15cc1a30efb/volumes"
Oct 30 19:35:50.454122 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:50.454052    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt" event={"ID":"f88b8b61-5083-47c0-821d-92bf69941379","Type":"ContainerStarted","Data":"14e57a57596a92cd3ac3e229a331d29e446591eed22b613a2a9b3db7957e079a"}
Oct 30 19:35:50.454704 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:50.454165    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"
Oct 30 19:35:50.489402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:50.489346    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt" podStartSLOduration=1.969702466 podStartE2EDuration="2.489331752s" totalImagesPullingTime="519.629286ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:50.487687096 +0000 UTC m=+1782.228678209" watchObservedRunningTime="2026-10-30 19:35:50.489331752 +0000 UTC m=+1782.230322851"
Oct 30 19:35:50.596672 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:50.596640    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64"]
Oct 30 19:35:50.597429 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:50.597411    4524 cpu_manager.go:414] "containerMap: removing container" podUID="968cf343-2e3b-4186-875d-2275e313a4b3" containerName="istio-proxy"
Oct 30 19:35:50.597429 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:50.597427    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="968cf343-2e3b-4186-875d-2275e313a4b3" containerName="istio-proxy"
Oct 30 19:35:50.597597 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:50.597556    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="968cf343-2e3b-4186-875d-2275e313a4b3" containerName="istio-proxy"
Oct 30 19:35:50.601541 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:50.601521    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64"
Oct 30 19:35:50.612383 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:50.612165    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w4-8e6762-dockercfg-rq5f2\""
Oct 30 19:35:50.615882 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:50.614502    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64"]
Oct 30 19:35:50.625407 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:50.625386    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w4-8e6762\""
Oct 30 19:35:50.735250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:50.730420    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d596f44-3f24-4f49-8baf-6dfe09ad625a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64\" (UID: \"5d596f44-3f24-4f49-8baf-6dfe09ad625a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64"
Oct 30 19:35:50.735250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:50.730842    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh2sl\" (UniqueName: \"kubernetes.io/projected/5d596f44-3f24-4f49-8baf-6dfe09ad625a-kube-api-access-bh2sl\") pod \"payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64\" (UID: \"5d596f44-3f24-4f49-8baf-6dfe09ad625a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64"
Oct 30 19:35:50.843360 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:50.839283    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh2sl\" (UniqueName: \"kubernetes.io/projected/5d596f44-3f24-4f49-8baf-6dfe09ad625a-kube-api-access-bh2sl\") pod \"payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64\" (UID: \"5d596f44-3f24-4f49-8baf-6dfe09ad625a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64"
Oct 30 19:35:50.843360 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:50.839450    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d596f44-3f24-4f49-8baf-6dfe09ad625a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64\" (UID: \"5d596f44-3f24-4f49-8baf-6dfe09ad625a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64"
Oct 30 19:35:50.843360 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:50.840683    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d596f44-3f24-4f49-8baf-6dfe09ad625a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64\" (UID: \"5d596f44-3f24-4f49-8baf-6dfe09ad625a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64"
Oct 30 19:35:50.904533 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:50.904208    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh2sl\" (UniqueName: \"kubernetes.io/projected/5d596f44-3f24-4f49-8baf-6dfe09ad625a-kube-api-access-bh2sl\") pod \"payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64\" (UID: \"5d596f44-3f24-4f49-8baf-6dfe09ad625a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64"
Oct 30 19:35:50.926725 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:50.926192    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64"
Oct 30 19:35:51.271212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:51.271160    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64"]
Oct 30 19:35:51.275767 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:35:51.275735    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d596f44_3f24_4f49_8baf_6dfe09ad625a.slice/crio-a356a966cfc08270649ff718d2fe7a243da30d2c89f833de015cabd052aa0596 WatchSource:0}: Error finding container a356a966cfc08270649ff718d2fe7a243da30d2c89f833de015cabd052aa0596: Status 404 returned error can't find the container with id a356a966cfc08270649ff718d2fe7a243da30d2c89f833de015cabd052aa0596
Oct 30 19:35:51.466501 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:51.466463    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64" event={"ID":"5d596f44-3f24-4f49-8baf-6dfe09ad625a","Type":"ContainerStarted","Data":"a356a966cfc08270649ff718d2fe7a243da30d2c89f833de015cabd052aa0596"}
Oct 30 19:35:52.475437 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:52.475399    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64" event={"ID":"5d596f44-3f24-4f49-8baf-6dfe09ad625a","Type":"ContainerStarted","Data":"5dd78c97dcc0d3fa421d008ba41d3d9d00422138d235ed9a6188c5a9669eda52"}
Oct 30 19:35:52.475886 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:52.475509    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64"
Oct 30 19:35:52.495190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:52.495133    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64" podStartSLOduration=2.495118795 podStartE2EDuration="2.495118795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:52.492564822 +0000 UTC m=+1784.233555902" watchObservedRunningTime="2026-10-30 19:35:52.495118795 +0000 UTC m=+1784.236109874"
Oct 30 19:35:55.487834 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:55.487799    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9"]
Oct 30 19:35:55.494334 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:55.494311    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9"
Oct 30 19:35:55.501803 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:55.501772    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-fd7c77-dockercfg-97w2g\""
Oct 30 19:35:55.504853 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:55.504552    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9"]
Oct 30 19:35:55.573274 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:55.573248    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-fd7c77\""
Oct 30 19:35:55.608193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:55.601150    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvj8h\" (UniqueName: \"kubernetes.io/projected/7ad49a59-872d-48c0-91f7-84a249089008-kube-api-access-kvj8h\") pod \"payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9\" (UID: \"7ad49a59-872d-48c0-91f7-84a249089008\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9"
Oct 30 19:35:55.608193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:55.601387    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ad49a59-872d-48c0-91f7-84a249089008-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9\" (UID: \"7ad49a59-872d-48c0-91f7-84a249089008\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9"
Oct 30 19:35:55.710190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:55.709247    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ad49a59-872d-48c0-91f7-84a249089008-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9\" (UID: \"7ad49a59-872d-48c0-91f7-84a249089008\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9"
Oct 30 19:35:55.710190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:55.709476    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvj8h\" (UniqueName: \"kubernetes.io/projected/7ad49a59-872d-48c0-91f7-84a249089008-kube-api-access-kvj8h\") pod \"payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9\" (UID: \"7ad49a59-872d-48c0-91f7-84a249089008\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9"
Oct 30 19:35:55.729193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:55.710858    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ad49a59-872d-48c0-91f7-84a249089008-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9\" (UID: \"7ad49a59-872d-48c0-91f7-84a249089008\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9"
Oct 30 19:35:55.769248 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:55.763232    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvj8h\" (UniqueName: \"kubernetes.io/projected/7ad49a59-872d-48c0-91f7-84a249089008-kube-api-access-kvj8h\") pod \"payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9\" (UID: \"7ad49a59-872d-48c0-91f7-84a249089008\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9"
Oct 30 19:35:55.846235 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:55.842566    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9"
Oct 30 19:35:56.201277 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:56.201246    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9"]
Oct 30 19:35:56.548029 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:56.523328    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"
Oct 30 19:35:56.653466 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:56.653422    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9" event={"ID":"7ad49a59-872d-48c0-91f7-84a249089008","Type":"ContainerStarted","Data":"086886a9bcb562dc32ce05dddcea0e7432333be1733158f058277c06338aae9e"}
Oct 30 19:35:56.653754 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:56.653717    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9" event={"ID":"7ad49a59-872d-48c0-91f7-84a249089008","Type":"ContainerStarted","Data":"8d8d99a32b86ddcd18cb64a8e4afca46ef505c3d2a63c0fd0f2f62a6dc209501"}
Oct 30 19:35:56.655218 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:56.655201    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9"
Oct 30 19:35:56.707152 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:35:56.707057    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9" podStartSLOduration=1.707034143 podStartE2EDuration="1.707034143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:56.68954495 +0000 UTC m=+1788.430536043" watchObservedRunningTime="2026-10-30 19:35:56.707034143 +0000 UTC m=+1788.448025223"
Oct 30 19:36:03.487799 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:03.487760    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64"
Oct 30 19:36:06.566812 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:06.566773    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"]
Oct 30 19:36:06.587287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:06.586389    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"]
Oct 30 19:36:06.587606 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:06.587586    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"
Oct 30 19:36:06.594971 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:06.594734    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w4-8e6762-serving-cert\""
Oct 30 19:36:06.694906 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:06.691116    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8thd\" (UniqueName: \"kubernetes.io/projected/067cde38-0f2e-432e-b278-415e4ce5ff86-kube-api-access-p8thd\") pod \"maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk\" (UID: \"067cde38-0f2e-432e-b278-415e4ce5ff86\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"
Oct 30 19:36:06.694906 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:06.691298    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/067cde38-0f2e-432e-b278-415e4ce5ff86-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk\" (UID: \"067cde38-0f2e-432e-b278-415e4ce5ff86\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"
Oct 30 19:36:06.694906 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:06.691343    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/067cde38-0f2e-432e-b278-415e4ce5ff86-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk\" (UID: \"067cde38-0f2e-432e-b278-415e4ce5ff86\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"
Oct 30 19:36:06.694906 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:06.691668    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/067cde38-0f2e-432e-b278-415e4ce5ff86-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk\" (UID: \"067cde38-0f2e-432e-b278-415e4ce5ff86\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"
Oct 30 19:36:06.792672 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:06.792637    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/067cde38-0f2e-432e-b278-415e4ce5ff86-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk\" (UID: \"067cde38-0f2e-432e-b278-415e4ce5ff86\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"
Oct 30 19:36:06.792875 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:06.792718    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8thd\" (UniqueName: \"kubernetes.io/projected/067cde38-0f2e-432e-b278-415e4ce5ff86-kube-api-access-p8thd\") pod \"maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk\" (UID: \"067cde38-0f2e-432e-b278-415e4ce5ff86\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"
Oct 30 19:36:06.792875 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:06.792744    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/067cde38-0f2e-432e-b278-415e4ce5ff86-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk\" (UID: \"067cde38-0f2e-432e-b278-415e4ce5ff86\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"
Oct 30 19:36:06.792875 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:06.792763    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/067cde38-0f2e-432e-b278-415e4ce5ff86-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk\" (UID: \"067cde38-0f2e-432e-b278-415e4ce5ff86\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"
Oct 30 19:36:06.793597 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:06.793573    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/067cde38-0f2e-432e-b278-415e4ce5ff86-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk\" (UID: \"067cde38-0f2e-432e-b278-415e4ce5ff86\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"
Oct 30 19:36:06.795729 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:06.795699    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/067cde38-0f2e-432e-b278-415e4ce5ff86-metrics-certs\") pod \"maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk\" (UID: \"067cde38-0f2e-432e-b278-415e4ce5ff86\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"
Oct 30 19:36:06.795829 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:06.795731    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/067cde38-0f2e-432e-b278-415e4ce5ff86-maas-api-tls\") pod \"maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk\" (UID: \"067cde38-0f2e-432e-b278-415e4ce5ff86\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"
Oct 30 19:36:06.804461 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:06.804436    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8thd\" (UniqueName: \"kubernetes.io/projected/067cde38-0f2e-432e-b278-415e4ce5ff86-kube-api-access-p8thd\") pod \"maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk\" (UID: \"067cde38-0f2e-432e-b278-415e4ce5ff86\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"
Oct 30 19:36:06.915691 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:06.915654    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"
Oct 30 19:36:07.353238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:07.334957    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"]
Oct 30 19:36:07.845838 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:07.845805    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk" event={"ID":"067cde38-0f2e-432e-b278-415e4ce5ff86","Type":"ContainerStarted","Data":"3ec2e2e72225167faa011d9cdf5408e27237247fa7696bdfa7ace63ea0b82390"}
Oct 30 19:36:08.709259 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:08.705103    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9"
Oct 30 19:36:08.865477 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:08.865443    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk" event={"ID":"067cde38-0f2e-432e-b278-415e4ce5ff86","Type":"ContainerStarted","Data":"b8bad8cd4690ec5e3a8ce00d377cac49d15254e5e285268d0ac594d363a98c32"}
Oct 30 19:36:08.865929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:08.865513    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"
Oct 30 19:36:09.690035 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:09.689999    4524 scope.go:122] "RemoveContainer" containerID="c8a1662c12c34bc51ef7769c12adac05796d87a567b0c72d4fa46f02224c5d6c"
Oct 30 19:36:12.243564 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:12.243499    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk" podStartSLOduration=5.726856036 podStartE2EDuration="6.243479142s" totalImagesPullingTime="516.623106ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:08.906083442 +0000 UTC m=+1800.647074522" watchObservedRunningTime="2026-10-30 19:36:12.243479142 +0000 UTC m=+1803.984470223"
Oct 30 19:36:12.244629 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:12.244592    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"]
Oct 30 19:36:12.252683 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:12.252654    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"
Oct 30 19:36:12.257494 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:12.257467    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-fd7c77-serving-cert\""
Oct 30 19:36:12.264553 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:12.264514    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"]
Oct 30 19:36:12.382211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:12.380681    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh\" (UID: \"914487e6-51d5-4f89-9aa7-e6cd7f329a3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"
Oct 30 19:36:12.382211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:12.380776    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8phnv\" (UniqueName: \"kubernetes.io/projected/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-kube-api-access-8phnv\") pod \"maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh\" (UID: \"914487e6-51d5-4f89-9aa7-e6cd7f329a3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"
Oct 30 19:36:12.382211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:12.380846    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh\" (UID: \"914487e6-51d5-4f89-9aa7-e6cd7f329a3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"
Oct 30 19:36:12.382211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:12.381003    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh\" (UID: \"914487e6-51d5-4f89-9aa7-e6cd7f329a3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"
Oct 30 19:36:12.482528 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:12.482489    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh\" (UID: \"914487e6-51d5-4f89-9aa7-e6cd7f329a3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"
Oct 30 19:36:12.482731 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:12.482549    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8phnv\" (UniqueName: \"kubernetes.io/projected/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-kube-api-access-8phnv\") pod \"maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh\" (UID: \"914487e6-51d5-4f89-9aa7-e6cd7f329a3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"
Oct 30 19:36:12.482731 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:12.482586    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh\" (UID: \"914487e6-51d5-4f89-9aa7-e6cd7f329a3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"
Oct 30 19:36:12.482731 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:12.482661    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh\" (UID: \"914487e6-51d5-4f89-9aa7-e6cd7f329a3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"
Oct 30 19:36:12.483531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:12.483501    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh\" (UID: \"914487e6-51d5-4f89-9aa7-e6cd7f329a3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"
Oct 30 19:36:12.485461 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:12.485425    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh\" (UID: \"914487e6-51d5-4f89-9aa7-e6cd7f329a3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"
Oct 30 19:36:12.485461 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:12.485461    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh\" (UID: \"914487e6-51d5-4f89-9aa7-e6cd7f329a3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"
Oct 30 19:36:12.494133 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:12.494067    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8phnv\" (UniqueName: \"kubernetes.io/projected/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-kube-api-access-8phnv\") pod \"maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh\" (UID: \"914487e6-51d5-4f89-9aa7-e6cd7f329a3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"
Oct 30 19:36:12.588828 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:12.588801    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"
Oct 30 19:36:13.023210 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:13.018149    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"]
Oct 30 19:36:13.972163 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:13.972121    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh" event={"ID":"914487e6-51d5-4f89-9aa7-e6cd7f329a3e","Type":"ContainerStarted","Data":"b8fb7132dccd04e2758ce517fa52b112532fdd2c84a814584c700c7e2b9bcab3"}
Oct 30 19:36:13.972586 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:13.972218    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh" event={"ID":"914487e6-51d5-4f89-9aa7-e6cd7f329a3e","Type":"ContainerStarted","Data":"2fa7e34e104834ab4356ef2178e8ebfe25d3589c35f54987620362388ff658ab"}
Oct 30 19:36:13.972586 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:13.972245    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"
Oct 30 19:36:13.993970 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:13.993913    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh" podStartSLOduration=1.592819185 podStartE2EDuration="1.993896622s" totalImagesPullingTime="401.077437ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:13.990351969 +0000 UTC m=+1805.731343060" watchObservedRunningTime="2026-10-30 19:36:13.993896622 +0000 UTC m=+1805.734887703"
Oct 30 19:36:14.915397 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:14.915356    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"
Oct 30 19:36:15.744617 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:36:15.744579    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:36:19.996848 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:19.996815    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"
Oct 30 19:36:25.454008 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.453974    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"]
Oct 30 19:36:25.469275 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.469187    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.481152 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.471096    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"]
Oct 30 19:36:25.535400 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.535367    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w6-0a6e66-openshift-default-dockercfg-9mmqk\""
Oct 30 19:36:25.545736 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.545706    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d30e03c9-c264-4b28-96de-720ac0cc24ba-istiod-ca-cert\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.545870 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.545810    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-envoy\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.545870 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.545848    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-workload-certs\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.545993 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.545921    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-workload-socket\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.545993 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.545978    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-token\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.546139 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.546121    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmmg7\" (UniqueName: \"kubernetes.io/projected/d30e03c9-c264-4b28-96de-720ac0cc24ba-kube-api-access-pmmg7\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.546236 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.546222    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-data\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.546300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.546246    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-credential-socket\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.546363 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.546337    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-podinfo\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.606431 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.606395    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"]
Oct 30 19:36:25.611877 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.611851    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.624686 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.624667    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"]
Oct 30 19:36:25.635538 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.635516    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-76117c-openshift-default-dockercfg-4nkjx\""
Oct 30 19:36:25.647483 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.647452    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-envoy\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.647631 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.647606    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-data\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.647717 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.647672    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmmg7\" (UniqueName: \"kubernetes.io/projected/d30e03c9-c264-4b28-96de-720ac0cc24ba-kube-api-access-pmmg7\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.647951 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.647914    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v8z6\" (UniqueName: \"kubernetes.io/projected/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-kube-api-access-8v8z6\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.648047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.648005    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-data\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.648120 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.648045    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-credential-socket\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.648120 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.648102    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-podinfo\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.648248 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.648150    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-workload-socket\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.648248 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.648228    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-workload-certs\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.648370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.648259    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d30e03c9-c264-4b28-96de-720ac0cc24ba-istiod-ca-cert\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.648370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.648298    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-token\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.648370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.648329    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istiod-ca-cert\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.648536 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.648371    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-podinfo\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.648536 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.648420    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-envoy\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.648536 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.648453    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-workload-certs\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.648536 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.648487    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-credential-socket\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.648536 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.648526    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-workload-socket\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.648810 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.648550    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-token\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.648810 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.648731    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-data\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.648810 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.648771    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-credential-socket\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.649253 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.649227    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-workload-socket\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.649537 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.649511    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-workload-certs\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.650048 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.650022    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d30e03c9-c264-4b28-96de-720ac0cc24ba-istiod-ca-cert\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.651429 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.651408    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-podinfo\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.651559 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.651545    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-envoy\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.660022 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.659999    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmmg7\" (UniqueName: \"kubernetes.io/projected/d30e03c9-c264-4b28-96de-720ac0cc24ba-kube-api-access-pmmg7\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.660201 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.660146    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-token\") pod \"e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") " pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.749764 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.749679    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-workload-socket\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.749764 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.749754    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-workload-certs\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.750003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.749803    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-token\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.750003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.749853    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istiod-ca-cert\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.750003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.749902    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-podinfo\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.750003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.749971    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-credential-socket\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.750276 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.750029    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-workload-socket\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.750276 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.750071    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-envoy\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.750276 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.750148    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-workload-certs\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.750276 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.750163    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-data\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.750276 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.750245    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v8z6\" (UniqueName: \"kubernetes.io/projected/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-kube-api-access-8v8z6\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.750529 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.750347    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-credential-socket\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.750573 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.750547    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-data\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.750743 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.750722    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istiod-ca-cert\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.752984 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.752957    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-envoy\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.753209 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.753193    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-podinfo\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.761862 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.761839    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-token\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.762362 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.762344    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v8z6\" (UniqueName: \"kubernetes.io/projected/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-kube-api-access-8v8z6\") pod \"e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") " pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:25.804584 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.804560    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:25.930442 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:25.930242    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:26.144027 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:36:26.143962    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd30e03c9_c264_4b28_96de_720ac0cc24ba.slice/crio-0e2f2224991cd563f2e17c50b9316bf93481e3f677460f72ae15ced67072241b WatchSource:0}: Error finding container 0e2f2224991cd563f2e17c50b9316bf93481e3f677460f72ae15ced67072241b: Status 404 returned error can't find the container with id 0e2f2224991cd563f2e17c50b9316bf93481e3f677460f72ae15ced67072241b
Oct 30 19:36:26.152268 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:26.150756    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:36:26.152268 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:26.150840    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:36:26.152268 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:26.150887    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:36:26.181477 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:26.180969    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"]
Oct 30 19:36:26.215299 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:26.215213    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w" event={"ID":"d30e03c9-c264-4b28-96de-720ac0cc24ba","Type":"ContainerStarted","Data":"0e2f2224991cd563f2e17c50b9316bf93481e3f677460f72ae15ced67072241b"}
Oct 30 19:36:26.241797 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:26.241769    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"]
Oct 30 19:36:26.249605 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:36:26.249579    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8b1d71f_87d1_4743_b192_ef5219a8c0f2.slice/crio-9cdb1deaae8a2aa2b474e8a04fe8e1127fdac434418078f05ad2497afe39025b WatchSource:0}: Error finding container 9cdb1deaae8a2aa2b474e8a04fe8e1127fdac434418078f05ad2497afe39025b: Status 404 returned error can't find the container with id 9cdb1deaae8a2aa2b474e8a04fe8e1127fdac434418078f05ad2497afe39025b
Oct 30 19:36:26.252408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:26.252374    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:36:26.252510 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:26.252457    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:36:26.252510 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:26.252504    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:36:27.243853 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:27.243816    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm" event={"ID":"e8b1d71f-87d1-4743-b192-ef5219a8c0f2","Type":"ContainerStarted","Data":"fc5524f96c965f1afecabd31721a287f29e943a3493ef65bc83cc222879ccb7e"}
Oct 30 19:36:27.244314 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:27.243876    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm" event={"ID":"e8b1d71f-87d1-4743-b192-ef5219a8c0f2","Type":"ContainerStarted","Data":"9cdb1deaae8a2aa2b474e8a04fe8e1127fdac434418078f05ad2497afe39025b"}
Oct 30 19:36:27.246206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:27.246165    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w" event={"ID":"d30e03c9-c264-4b28-96de-720ac0cc24ba","Type":"ContainerStarted","Data":"c9ba90746ea8f06bab9e3b971ebb7e5a87022d6ea91ca9289a99f550c6752951"}
Oct 30 19:36:27.295194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:27.281522    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm" podStartSLOduration=2.2814979109999998 podStartE2EDuration="2.281497911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:27.266716549 +0000 UTC m=+1819.007707628" watchObservedRunningTime="2026-10-30 19:36:27.281497911 +0000 UTC m=+1819.022488992"
Oct 30 19:36:27.338193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:27.337613    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w" podStartSLOduration=2.337591308 podStartE2EDuration="2.337591308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:27.294830086 +0000 UTC m=+1819.035821166" watchObservedRunningTime="2026-10-30 19:36:27.337591308 +0000 UTC m=+1819.078582388"
Oct 30 19:36:28.050346 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:28.050313    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:28.050648 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:28.050626    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:28.065442 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:28.065407    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused" start-of-body=
Oct 30 19:36:28.109038 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:28.108991    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w" podUID="d30e03c9-c264-4b28-96de-720ac0cc24ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused"
Oct 30 19:36:28.109546 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:28.065765    4524 patch_prober.go:29] interesting pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused" start-of-body=
Oct 30 19:36:28.109733 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:28.109703    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm" podUID="e8b1d71f-87d1-4743-b192-ef5219a8c0f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused"
Oct 30 19:36:28.807197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:28.806460    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused" start-of-body=
Oct 30 19:36:28.807197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:28.806539    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w" podUID="d30e03c9-c264-4b28-96de-720ac0cc24ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused"
Oct 30 19:36:28.934564 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:28.934525    4524 patch_prober.go:29] interesting pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused" start-of-body=
Oct 30 19:36:28.934851 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:28.934820    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm" podUID="e8b1d71f-87d1-4743-b192-ef5219a8c0f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused"
Oct 30 19:36:29.806002 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:29.805535    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused" start-of-body=
Oct 30 19:36:29.806002 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:29.805599    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w" podUID="d30e03c9-c264-4b28-96de-720ac0cc24ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused"
Oct 30 19:36:29.949197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:29.941575    4524 patch_prober.go:29] interesting pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused" start-of-body=
Oct 30 19:36:29.949197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:29.941639    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm" podUID="e8b1d71f-87d1-4743-b192-ef5219a8c0f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused"
Oct 30 19:36:30.822629 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:30.822575    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused" start-of-body=
Oct 30 19:36:30.838010 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:30.837960    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w" podUID="d30e03c9-c264-4b28-96de-720ac0cc24ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused"
Oct 30 19:36:30.967194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:30.957614    4524 patch_prober.go:29] interesting pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused" start-of-body=
Oct 30 19:36:30.967194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:30.957676    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm" podUID="e8b1d71f-87d1-4743-b192-ef5219a8c0f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused"
Oct 30 19:36:31.813198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:31.811066    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused" start-of-body=
Oct 30 19:36:31.813198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:31.811124    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w" podUID="d30e03c9-c264-4b28-96de-720ac0cc24ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused"
Oct 30 19:36:31.937083 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:31.937045    4524 patch_prober.go:29] interesting pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused" start-of-body=
Oct 30 19:36:31.937402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:31.937369    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm" podUID="e8b1d71f-87d1-4743-b192-ef5219a8c0f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused"
Oct 30 19:36:32.158417 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.158372    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"]
Oct 30 19:36:32.163906 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.163878    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.249981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.248198    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"]
Oct 30 19:36:32.257229 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.257197    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-e3586f-openshift-default-dockercfg-czzb7\""
Oct 30 19:36:32.428666 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.428577    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-token\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.428908 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.428887    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-workload-certs\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.429047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.429029    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvcbf\" (UniqueName: \"kubernetes.io/projected/db5e3658-2a84-43ee-8390-e8f88ae56ea0-kube-api-access-jvcbf\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.429163 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.429149    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-workload-socket\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.429366 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.429347    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-credential-socket\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.429525 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.429510    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-data\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.429627 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.429613    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istiod-ca-cert\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.429800 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.429782    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-podinfo\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.430022 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.430003    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-envoy\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.548128 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.546946    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-token\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.548128 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.547003    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-workload-certs\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.548128 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.547044    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvcbf\" (UniqueName: \"kubernetes.io/projected/db5e3658-2a84-43ee-8390-e8f88ae56ea0-kube-api-access-jvcbf\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.548128 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.547076    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-workload-socket\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.548128 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.547195    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-credential-socket\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.548128 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.547275    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-data\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.548128 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.547298    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istiod-ca-cert\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.548128 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.547405    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-podinfo\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.548128 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.547561    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-envoy\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.548758 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.548662    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-credential-socket\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.549661 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.549626    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-workload-certs\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.574282 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.557639    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-workload-socket\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.574282 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.562349    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istiod-ca-cert\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.574282 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.562665    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-data\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.611440 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.610741    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-token\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.611440 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.610769    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-podinfo\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.611440 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.611366    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvcbf\" (UniqueName: \"kubernetes.io/projected/db5e3658-2a84-43ee-8390-e8f88ae56ea0-kube-api-access-jvcbf\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.611722 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.611559    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-envoy\") pod \"e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") " pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.817242 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.809471    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:32.817242 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.812854    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused" start-of-body=
Oct 30 19:36:32.817242 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.812921    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w" podUID="d30e03c9-c264-4b28-96de-720ac0cc24ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.156:15021/healthz/ready\": dial tcp 10.133.0.156:15021: connect: connection refused"
Oct 30 19:36:32.932351 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.931519    4524 patch_prober.go:29] interesting pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused" start-of-body=
Oct 30 19:36:32.932351 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:32.931588    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm" podUID="e8b1d71f-87d1-4743-b192-ef5219a8c0f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused"
Oct 30 19:36:33.349492 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:36:33.347854    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb5e3658_2a84_43ee_8390_e8f88ae56ea0.slice/crio-3e5a36a9100f8e5aaca42d075cd1882940f7890f13b45ae19145eb242ec75a1b WatchSource:0}: Error finding container 3e5a36a9100f8e5aaca42d075cd1882940f7890f13b45ae19145eb242ec75a1b: Status 404 returned error can't find the container with id 3e5a36a9100f8e5aaca42d075cd1882940f7890f13b45ae19145eb242ec75a1b
Oct 30 19:36:33.349492 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:33.348157    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"]
Oct 30 19:36:33.352619 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:33.352586    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:36:33.352732 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:33.352664    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:36:33.352732 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:33.352705    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:36:33.587194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:33.586789    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq" event={"ID":"db5e3658-2a84-43ee-8390-e8f88ae56ea0","Type":"ContainerStarted","Data":"3e5a36a9100f8e5aaca42d075cd1882940f7890f13b45ae19145eb242ec75a1b"}
Oct 30 19:36:33.959281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:33.953834    4524 patch_prober.go:29] interesting pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused" start-of-body=
Oct 30 19:36:33.959281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:33.953915    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm" podUID="e8b1d71f-87d1-4743-b192-ef5219a8c0f2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.157:15021/healthz/ready\": dial tcp 10.133.0.157:15021: connect: connection refused"
Oct 30 19:36:34.045818 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:34.045760    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:34.049299 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:34.047457    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:34.050279 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:34.050254    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:36:34.613840 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:34.613803    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq" event={"ID":"db5e3658-2a84-43ee-8390-e8f88ae56ea0","Type":"ContainerStarted","Data":"25426ac8e6379ca5da5c5198f7583be3867fde176eef78de076032ac5349154c"}
Oct 30 19:36:34.703264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:34.700875    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq" podStartSLOduration=2.70084984 podStartE2EDuration="2.70084984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:34.664522358 +0000 UTC m=+1826.405513441" watchObservedRunningTime="2026-10-30 19:36:34.70084984 +0000 UTC m=+1826.441840920"
Oct 30 19:36:34.815577 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:34.815538    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:34.819304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:34.819271    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused" start-of-body=
Oct 30 19:36:34.819517 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:34.819331    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq" podUID="db5e3658-2a84-43ee-8390-e8f88ae56ea0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused"
Oct 30 19:36:35.197191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:35.191770    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:35.197191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:35.193405    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:35.212238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:35.206732    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:36:35.815369 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:35.813725    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused" start-of-body=
Oct 30 19:36:35.815369 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:35.813790    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq" podUID="db5e3658-2a84-43ee-8390-e8f88ae56ea0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused"
Oct 30 19:36:36.828117 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:36.827982    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused" start-of-body=
Oct 30 19:36:36.828117 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:36.828060    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq" podUID="db5e3658-2a84-43ee-8390-e8f88ae56ea0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused"
Oct 30 19:36:37.814226 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:37.812795    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused" start-of-body=
Oct 30 19:36:37.814226 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:37.812853    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq" podUID="db5e3658-2a84-43ee-8390-e8f88ae56ea0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused"
Oct 30 19:36:38.814062 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:38.814024    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused" start-of-body=
Oct 30 19:36:38.814747 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:38.814692    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq" podUID="db5e3658-2a84-43ee-8390-e8f88ae56ea0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused"
Oct 30 19:36:39.819193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:39.819046    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused" start-of-body=
Oct 30 19:36:39.819193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:39.819109    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq" podUID="db5e3658-2a84-43ee-8390-e8f88ae56ea0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.158:15021/healthz/ready\": dial tcp 10.133.0.158:15021: connect: connection refused"
Oct 30 19:36:41.171766 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:41.171730    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:41.172424 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:41.172400    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:41.173453 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:41.173431    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:36:45.628383 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:36:45.628329    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:36:54.767437 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:54.767349    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"]
Oct 30 19:36:54.781382 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:54.781353    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"
Oct 30 19:36:54.788881 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:54.788851    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"]
Oct 30 19:36:54.791820 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:54.789239    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w4-1b1531-serving-cert\""
Oct 30 19:36:54.884309 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:54.884278    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3f46492-0c06-4e9d-bd74-793bee10103c-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8\" (UID: \"c3f46492-0c06-4e9d-bd74-793bee10103c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"
Oct 30 19:36:54.884494 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:54.884356    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3f46492-0c06-4e9d-bd74-793bee10103c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8\" (UID: \"c3f46492-0c06-4e9d-bd74-793bee10103c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"
Oct 30 19:36:54.884494 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:54.884480    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnnn9\" (UniqueName: \"kubernetes.io/projected/c3f46492-0c06-4e9d-bd74-793bee10103c-kube-api-access-nnnn9\") pod \"maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8\" (UID: \"c3f46492-0c06-4e9d-bd74-793bee10103c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"
Oct 30 19:36:54.884638 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:54.884537    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3f46492-0c06-4e9d-bd74-793bee10103c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8\" (UID: \"c3f46492-0c06-4e9d-bd74-793bee10103c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"
Oct 30 19:36:55.029762 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:55.018211    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3f46492-0c06-4e9d-bd74-793bee10103c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8\" (UID: \"c3f46492-0c06-4e9d-bd74-793bee10103c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"
Oct 30 19:36:55.029762 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:55.018362    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnnn9\" (UniqueName: \"kubernetes.io/projected/c3f46492-0c06-4e9d-bd74-793bee10103c-kube-api-access-nnnn9\") pod \"maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8\" (UID: \"c3f46492-0c06-4e9d-bd74-793bee10103c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"
Oct 30 19:36:55.029762 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:55.018430    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3f46492-0c06-4e9d-bd74-793bee10103c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8\" (UID: \"c3f46492-0c06-4e9d-bd74-793bee10103c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"
Oct 30 19:36:55.029762 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:55.018633    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3f46492-0c06-4e9d-bd74-793bee10103c-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8\" (UID: \"c3f46492-0c06-4e9d-bd74-793bee10103c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"
Oct 30 19:36:55.029762 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:55.024074    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3f46492-0c06-4e9d-bd74-793bee10103c-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8\" (UID: \"c3f46492-0c06-4e9d-bd74-793bee10103c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"
Oct 30 19:36:55.029762 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:55.026654    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3f46492-0c06-4e9d-bd74-793bee10103c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8\" (UID: \"c3f46492-0c06-4e9d-bd74-793bee10103c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"
Oct 30 19:36:55.040931 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:55.040237    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnnn9\" (UniqueName: \"kubernetes.io/projected/c3f46492-0c06-4e9d-bd74-793bee10103c-kube-api-access-nnnn9\") pod \"maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8\" (UID: \"c3f46492-0c06-4e9d-bd74-793bee10103c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"
Oct 30 19:36:55.049133 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:55.049098    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3f46492-0c06-4e9d-bd74-793bee10103c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8\" (UID: \"c3f46492-0c06-4e9d-bd74-793bee10103c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"
Oct 30 19:36:55.110333 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:55.110296    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"
Oct 30 19:36:55.606188 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:55.600565    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"]
Oct 30 19:36:56.032527 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:56.032445    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8" event={"ID":"c3f46492-0c06-4e9d-bd74-793bee10103c","Type":"ContainerStarted","Data":"c86e697bc22baf5c4ecd8d3881b7e524c62c3a6e621a8b03b081368af2cd9a22"}
Oct 30 19:36:57.051639 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:57.051532    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8" event={"ID":"c3f46492-0c06-4e9d-bd74-793bee10103c","Type":"ContainerStarted","Data":"66f6bf2a143ef966535dcd51d085f684ee1fa62e502c3ee22fae54203d55d3dd"}
Oct 30 19:36:57.052814 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:57.052788    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"
Oct 30 19:36:57.084006 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:36:57.083954    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8" podStartSLOduration=2.605402473 podStartE2EDuration="3.083942346s" totalImagesPullingTime="478.539873ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:57.077556963 +0000 UTC m=+1848.818548042" watchObservedRunningTime="2026-10-30 19:36:57.083942346 +0000 UTC m=+1848.824933425"
Oct 30 19:37:04.073982 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:04.073951    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"
Oct 30 19:37:09.765952 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:09.765919    4524 scope.go:122] "RemoveContainer" containerID="e9e21d4e476d3aca7bbdb5e19995e7ea39984cc1bb701e3e3c09806da21dcdd4"
Oct 30 19:37:15.628361 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:37:15.628319    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:37:16.538467 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:16.538433    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"]
Oct 30 19:37:20.212356 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.212326    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"]
Oct 30 19:37:20.217531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.217506    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.243911 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.243884    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"]
Oct 30 19:37:20.308203 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.308153    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-fd7c77-openshift-default-dockercfg-9t69p\""
Oct 30 19:37:20.414239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.414206    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-workload-certs\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.414400 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.414255    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istiod-ca-cert\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.414400 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.414281    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-envoy\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.414484 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.414420    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-workload-socket\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.414523 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.414482    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-podinfo\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.414588 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.414566    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-data\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.414640 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.414625    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-token\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.414751 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.414735    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngq7c\" (UniqueName: \"kubernetes.io/projected/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-kube-api-access-ngq7c\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.414814 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.414778    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-credential-socket\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.521253 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.521147    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-workload-certs\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.521253 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.521228    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istiod-ca-cert\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.521253 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.521252    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-envoy\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.521545 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.521301    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-workload-socket\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.521545 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.521355    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-podinfo\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.521545 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.521399    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-data\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.521545 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.521457    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-token\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.521766 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.521589    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngq7c\" (UniqueName: \"kubernetes.io/projected/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-kube-api-access-ngq7c\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.521766 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.521651    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-credential-socket\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.522575 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.521894    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-workload-socket\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.522575 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.522094    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-credential-socket\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.522575 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.522130    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-workload-certs\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.522827 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.522702    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-data\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.523389 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.523335    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istiod-ca-cert\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.530894 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.530721    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-podinfo\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.539671 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.539572    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-token\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.539910 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.539880    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-envoy\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.540586 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.540559    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngq7c\" (UniqueName: \"kubernetes.io/projected/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-kube-api-access-ngq7c\") pod \"e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") " pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.546797 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.546593    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:20.892793 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.892151    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"]
Oct 30 19:37:20.902833 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:37:20.902804    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb3f6371_c4db_4bd9_8101_b34064bb6cc7.slice/crio-e545117a2cb28a7c056aa7d60a7e32095e94fbcee9d194cce7b3c119f7157e8e WatchSource:0}: Error finding container e545117a2cb28a7c056aa7d60a7e32095e94fbcee9d194cce7b3c119f7157e8e: Status 404 returned error can't find the container with id e545117a2cb28a7c056aa7d60a7e32095e94fbcee9d194cce7b3c119f7157e8e
Oct 30 19:37:20.906949 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.906910    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:37:20.907156 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.907132    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:37:20.907368 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:20.907346    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:37:21.425399 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:21.425359    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d" event={"ID":"fb3f6371-c4db-4bd9-8101-b34064bb6cc7","Type":"ContainerStarted","Data":"20ffa972d1f2f04cfbff37644ef557c1cd6e46d135c88b4abcc252d7844e74a4"}
Oct 30 19:37:21.425797 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:21.425457    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d" event={"ID":"fb3f6371-c4db-4bd9-8101-b34064bb6cc7","Type":"ContainerStarted","Data":"e545117a2cb28a7c056aa7d60a7e32095e94fbcee9d194cce7b3c119f7157e8e"}
Oct 30 19:37:21.450251 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:21.450196    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d" podStartSLOduration=1.450164895 podStartE2EDuration="1.450164895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:21.448646163 +0000 UTC m=+1873.189637240" watchObservedRunningTime="2026-10-30 19:37:21.450164895 +0000 UTC m=+1873.191155973"
Oct 30 19:37:21.555360 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:21.554853    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:21.568835 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:21.568462    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:37:21.568835 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:21.568551    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d" podUID="fb3f6371-c4db-4bd9-8101-b34064bb6cc7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 19:37:22.575192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:22.571544    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:37:22.575192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:22.571618    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d" podUID="fb3f6371-c4db-4bd9-8101-b34064bb6cc7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 19:37:23.546910 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:23.546877    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:37:23.547115 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:23.546940    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d" podUID="fb3f6371-c4db-4bd9-8101-b34064bb6cc7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 19:37:24.547695 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:24.547661    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:37:24.548090 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:24.547729    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d" podUID="fb3f6371-c4db-4bd9-8101-b34064bb6cc7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 19:37:25.546972 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:25.546930    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:37:25.547205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:25.546988    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d" podUID="fb3f6371-c4db-4bd9-8101-b34064bb6cc7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 19:37:26.554205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:26.553699    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:37:26.554205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:26.553774    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d" podUID="fb3f6371-c4db-4bd9-8101-b34064bb6cc7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 19:37:27.547032 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:27.546990    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused" start-of-body=
Oct 30 19:37:27.547537 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:27.547051    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d" podUID="fb3f6371-c4db-4bd9-8101-b34064bb6cc7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.160:15021/healthz/ready\": dial tcp 10.133.0.160:15021: connect: connection refused"
Oct 30 19:37:28.552351 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:28.552320    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:28.552841 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:28.552742    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:28.553531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:28.553511    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:37:29.759934 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.759899    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"]
Oct 30 19:37:29.773669 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.773635    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"]
Oct 30 19:37:29.773817 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.773761    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.782489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.782458    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w4-8e6762-openshift-default-dockercfg-ns2mw\""
Oct 30 19:37:29.819068 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.819010    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-data\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.820016 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.819083    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b05d10a-5c87-42d8-8880-2511d27bf716-istiod-ca-cert\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.820016 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.819165    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-podinfo\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.820016 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.819347    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-workload-certs\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.820016 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.819389    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-workload-socket\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.820016 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.819411    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-envoy\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.820016 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.819440    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-credential-socket\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.820016 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.819509    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-token\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.820016 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.819602    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st2kj\" (UniqueName: \"kubernetes.io/projected/1b05d10a-5c87-42d8-8880-2511d27bf716-kube-api-access-st2kj\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.921003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.920960    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-workload-certs\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.921003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.921013    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-workload-socket\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.921293 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.921032    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-envoy\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.921293 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.921055    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-credential-socket\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.921293 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.921115    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-token\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.921293 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.921214    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st2kj\" (UniqueName: \"kubernetes.io/projected/1b05d10a-5c87-42d8-8880-2511d27bf716-kube-api-access-st2kj\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.921293 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.921290    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-data\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.921584 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.921332    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b05d10a-5c87-42d8-8880-2511d27bf716-istiod-ca-cert\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.921584 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.921408    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-workload-socket\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.921584 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.921421    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-podinfo\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.921584 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.921515    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-workload-certs\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.921749 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.921669    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-credential-socket\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.921749 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.921722    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-data\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.922095 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.922073    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b05d10a-5c87-42d8-8880-2511d27bf716-istiod-ca-cert\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.923454 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.923426    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-envoy\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.923939 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.923920    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-podinfo\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.936579 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.936540    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-st2kj\" (UniqueName: \"kubernetes.io/projected/1b05d10a-5c87-42d8-8880-2511d27bf716-kube-api-access-st2kj\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:29.937586 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:29.937569    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-token\") pod \"e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") " pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:30.097649 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:30.097612    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:30.397962 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:30.397817    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"]
Oct 30 19:37:30.399532 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:30.399493    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:37:30.399641 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:30.399577    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:37:30.399641 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:30.399619    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:37:30.578076 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:30.578007    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq" event={"ID":"1b05d10a-5c87-42d8-8880-2511d27bf716","Type":"ContainerStarted","Data":"f868876d9b04b212155af1101e3ab6fd07a59665b55fe34a8caf57bc17f15305"}
Oct 30 19:37:31.593211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:31.592417    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt" podUID="930da0d5-fe44-4862-ad00-2c789c18882c" containerName="main" containerID="cri-o://f1ee090517aa66d91bee1d9bfd1d06c3ab84fcc47424ac93be6ef421b4db5e36" gracePeriod=45
Oct 30 19:37:31.612931 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:31.612889    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq" event={"ID":"1b05d10a-5c87-42d8-8880-2511d27bf716","Type":"ContainerStarted","Data":"ae5ced2f8ce1f3db2704d0becb437d38dd6f547113f9094b3b527bb4c75a10c5"}
Oct 30 19:37:31.640194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:31.640133    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq" podStartSLOduration=2.640116866 podStartE2EDuration="2.640116866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:31.637931133 +0000 UTC m=+1883.378922226" watchObservedRunningTime="2026-10-30 19:37:31.640116866 +0000 UTC m=+1883.381107945"
Oct 30 19:37:31.798193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:31.797143    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt" podUID="930da0d5-fe44-4862-ad00-2c789c18882c" containerName="main" probeResult="failure" output="Get \"https://10.133.0.146:8000/ready\": dial tcp 10.133.0.146:8000: connect: connection refused"
Oct 30 19:37:32.117108 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.117074    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:32.117476 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.117454    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.161:15021/healthz/ready\": dial tcp 10.133.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:37:32.117622 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.117598    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq" podUID="1b05d10a-5c87-42d8-8880-2511d27bf716" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.161:15021/healthz/ready\": dial tcp 10.133.0.161:15021: connect: connection refused"
Oct 30 19:37:32.383242 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.382518    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:37:32.417194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.397663    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/930da0d5-fe44-4862-ad00-2c789c18882c-tls-certs\" (UniqueName: \"kubernetes.io/secret/930da0d5-fe44-4862-ad00-2c789c18882c-tls-certs\") pod \"930da0d5-fe44-4862-ad00-2c789c18882c\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") "
Oct 30 19:37:32.417194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.397766    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/930da0d5-fe44-4862-ad00-2c789c18882c-kube-api-access-pc5rg\" (UniqueName: \"kubernetes.io/projected/930da0d5-fe44-4862-ad00-2c789c18882c-kube-api-access-pc5rg\") pod \"930da0d5-fe44-4862-ad00-2c789c18882c\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") "
Oct 30 19:37:32.417194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.397813    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-home\" (UniqueName: \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-home\") pod \"930da0d5-fe44-4862-ad00-2c789c18882c\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") "
Oct 30 19:37:32.417194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.397871    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-tmp-dir\") pod \"930da0d5-fe44-4862-ad00-2c789c18882c\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") "
Oct 30 19:37:32.417194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.398007    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-dshm\") pod \"930da0d5-fe44-4862-ad00-2c789c18882c\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") "
Oct 30 19:37:32.417194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.398092    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-model-cache\") pod \"930da0d5-fe44-4862-ad00-2c789c18882c\" (UID: \"930da0d5-fe44-4862-ad00-2c789c18882c\") "
Oct 30 19:37:32.417194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.416706    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-tmp-dir" pod "930da0d5-fe44-4862-ad00-2c789c18882c" (UID: "930da0d5-fe44-4862-ad00-2c789c18882c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:37:32.444192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.437450    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-home" pod "930da0d5-fe44-4862-ad00-2c789c18882c" (UID: "930da0d5-fe44-4862-ad00-2c789c18882c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:37:32.444192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.439904    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-model-cache" pod "930da0d5-fe44-4862-ad00-2c789c18882c" (UID: "930da0d5-fe44-4862-ad00-2c789c18882c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:37:32.501664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.501417    4524 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-model-cache\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:32.501664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.501458    4524 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-home\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:32.501664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.501471    4524 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-tmp-dir\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:32.528586 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.527501    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/930da0d5-fe44-4862-ad00-2c789c18882c-kube-api-access-pc5rg" pod "930da0d5-fe44-4862-ad00-2c789c18882c" (UID: "930da0d5-fe44-4862-ad00-2c789c18882c"). InnerVolumeSpecName "kube-api-access-pc5rg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:37:32.543225 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.532676    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/930da0d5-fe44-4862-ad00-2c789c18882c-tls-certs" pod "930da0d5-fe44-4862-ad00-2c789c18882c" (UID: "930da0d5-fe44-4862-ad00-2c789c18882c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:37:32.543225 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.539321    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-dshm" pod "930da0d5-fe44-4862-ad00-2c789c18882c" (UID: "930da0d5-fe44-4862-ad00-2c789c18882c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:37:32.623811 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.623769    4524 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/930da0d5-fe44-4862-ad00-2c789c18882c-tls-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:32.623811 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.623811    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pc5rg\" (UniqueName: \"kubernetes.io/projected/930da0d5-fe44-4862-ad00-2c789c18882c-kube-api-access-pc5rg\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:32.624394 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.623825    4524 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/930da0d5-fe44-4862-ad00-2c789c18882c-dshm\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:32.632190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.632017    4524 generic.go:356] "Generic (PLEG): container finished" podID="930da0d5-fe44-4862-ad00-2c789c18882c" containerID="f1ee090517aa66d91bee1d9bfd1d06c3ab84fcc47424ac93be6ef421b4db5e36" exitCode=0
Oct 30 19:37:32.635281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.633585    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"
Oct 30 19:37:32.635281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.634445    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt" event={"ID":"930da0d5-fe44-4862-ad00-2c789c18882c","Type":"ContainerDied","Data":"f1ee090517aa66d91bee1d9bfd1d06c3ab84fcc47424ac93be6ef421b4db5e36"}
Oct 30 19:37:32.635281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.634490    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt" event={"ID":"930da0d5-fe44-4862-ad00-2c789c18882c","Type":"ContainerDied","Data":"437f08d58d20241c55c590634869281e390e55172b5a6ae11188834672b02ee1"}
Oct 30 19:37:32.635281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.634512    4524 scope.go:122] "RemoveContainer" containerID="f1ee090517aa66d91bee1d9bfd1d06c3ab84fcc47424ac93be6ef421b4db5e36"
Oct 30 19:37:32.650722 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.650697    4524 scope.go:122] "RemoveContainer" containerID="f1ee090517aa66d91bee1d9bfd1d06c3ab84fcc47424ac93be6ef421b4db5e36"
Oct 30 19:37:32.651023 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:37:32.650997    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1ee090517aa66d91bee1d9bfd1d06c3ab84fcc47424ac93be6ef421b4db5e36\": container with ID starting with f1ee090517aa66d91bee1d9bfd1d06c3ab84fcc47424ac93be6ef421b4db5e36 not found: ID does not exist" containerID="f1ee090517aa66d91bee1d9bfd1d06c3ab84fcc47424ac93be6ef421b4db5e36"
Oct 30 19:37:32.651119 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.651026    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1ee090517aa66d91bee1d9bfd1d06c3ab84fcc47424ac93be6ef421b4db5e36"} err="failed to get container status \"f1ee090517aa66d91bee1d9bfd1d06c3ab84fcc47424ac93be6ef421b4db5e36\": rpc error: code = NotFound desc = could not find container \"f1ee090517aa66d91bee1d9bfd1d06c3ab84fcc47424ac93be6ef421b4db5e36\": container with ID starting with f1ee090517aa66d91bee1d9bfd1d06c3ab84fcc47424ac93be6ef421b4db5e36 not found: ID does not exist"
Oct 30 19:37:32.663891 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.663020    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"]
Oct 30 19:37:32.668029 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:32.668000    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-76117c/test-model-76117c-kserve-58cd949d8f-9tnxt"]
Oct 30 19:37:33.102220 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:33.098767    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.161:15021/healthz/ready\": dial tcp 10.133.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:37:33.102220 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:33.098823    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq" podUID="1b05d10a-5c87-42d8-8880-2511d27bf716" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.161:15021/healthz/ready\": dial tcp 10.133.0.161:15021: connect: connection refused"
Oct 30 19:37:34.110207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:34.107560    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.161:15021/healthz/ready\": dial tcp 10.133.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:37:34.110207 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:34.107629    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq" podUID="1b05d10a-5c87-42d8-8880-2511d27bf716" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.161:15021/healthz/ready\": dial tcp 10.133.0.161:15021: connect: connection refused"
Oct 30 19:37:34.407531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:34.407451    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="930da0d5-fe44-4862-ad00-2c789c18882c" path="/var/lib/kubelet/pods/930da0d5-fe44-4862-ad00-2c789c18882c/volumes"
Oct 30 19:37:35.099693 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:35.099657    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.161:15021/healthz/ready\": dial tcp 10.133.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:37:35.099967 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:35.099940    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq" podUID="1b05d10a-5c87-42d8-8880-2511d27bf716" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.161:15021/healthz/ready\": dial tcp 10.133.0.161:15021: connect: connection refused"
Oct 30 19:37:36.102225 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:36.100568    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.161:15021/healthz/ready\": dial tcp 10.133.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:37:36.102225 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:36.100627    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq" podUID="1b05d10a-5c87-42d8-8880-2511d27bf716" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.161:15021/healthz/ready\": dial tcp 10.133.0.161:15021: connect: connection refused"
Oct 30 19:37:37.114204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:37.114108    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.161:15021/healthz/ready\": dial tcp 10.133.0.161:15021: connect: connection refused" start-of-body=
Oct 30 19:37:37.114708 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:37.114203    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq" podUID="1b05d10a-5c87-42d8-8880-2511d27bf716" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.161:15021/healthz/ready\": dial tcp 10.133.0.161:15021: connect: connection refused"
Oct 30 19:37:38.209560 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:38.209528    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:38.211737 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:38.211715    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:38.212642 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:37:38.212622    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:37:45.631041 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:37:45.631001    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:38:09.799707 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:09.799676    4524 scope.go:122] "RemoveContainer" containerID="d79c4e372d503938a6c524b58c08f5ae0c9e450909bb2bcf845e6f73991b8319"
Oct 30 19:38:15.655949 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:38:15.655902    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:38:19.770670 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:19.770633    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"]
Oct 30 19:38:19.771159 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:19.770872    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm" podUID="e8b1d71f-87d1-4743-b192-ef5219a8c0f2" containerName="istio-proxy" containerID="cri-o://fc5524f96c965f1afecabd31721a287f29e943a3493ef65bc83cc222879ccb7e" gracePeriod=30
Oct 30 19:38:20.196136 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:20.196067    4524 patch_prober.go:29] interesting pod/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:38:20.196402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:20.196373    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm" podUID="e8b1d71f-87d1-4743-b192-ef5219a8c0f2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:38:25.046442 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.046078    4524 generic.go:356] "Generic (PLEG): container finished" podID="e8b1d71f-87d1-4743-b192-ef5219a8c0f2" containerID="fc5524f96c965f1afecabd31721a287f29e943a3493ef65bc83cc222879ccb7e" exitCode=0
Oct 30 19:38:25.046442 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.046162    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm" event={"ID":"e8b1d71f-87d1-4743-b192-ef5219a8c0f2","Type":"ContainerDied","Data":"fc5524f96c965f1afecabd31721a287f29e943a3493ef65bc83cc222879ccb7e"}
Oct 30 19:38:25.235712 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.235683    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:38:25.281265 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.281229    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"]
Oct 30 19:38:25.282530 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.282503    4524 cpu_manager.go:414] "containerMap: removing container" podUID="e8b1d71f-87d1-4743-b192-ef5219a8c0f2" containerName="istio-proxy"
Oct 30 19:38:25.282530 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.282530    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8b1d71f-87d1-4743-b192-ef5219a8c0f2" containerName="istio-proxy"
Oct 30 19:38:25.282721 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.282552    4524 cpu_manager.go:414] "containerMap: removing container" podUID="930da0d5-fe44-4862-ad00-2c789c18882c" containerName="main"
Oct 30 19:38:25.282721 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.282559    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="930da0d5-fe44-4862-ad00-2c789c18882c" containerName="main"
Oct 30 19:38:25.282817 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.282726    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="930da0d5-fe44-4862-ad00-2c789c18882c" containerName="main"
Oct 30 19:38:25.282817 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.282760    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8b1d71f-87d1-4743-b192-ef5219a8c0f2" containerName="istio-proxy"
Oct 30 19:38:25.287416 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.287385    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.296838 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.296777    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"]
Oct 30 19:38:25.297282 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.297262    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w4-1b1531-openshift-default-dockercfg-5ppz9\""
Oct 30 19:38:25.383536 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.383498    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-workload-socket\") pod \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") "
Oct 30 19:38:25.383702 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.383556    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istiod-ca-cert\") pod \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") "
Oct 30 19:38:25.383702 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.383607    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-envoy\") pod \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") "
Oct 30 19:38:25.383702 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.383641    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-credential-socket\") pod \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") "
Oct 30 19:38:25.383889 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.383710    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-token\" (UniqueName: \"kubernetes.io/projected/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-token\") pod \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") "
Oct 30 19:38:25.383889 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.383751    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-kube-api-access-8v8z6\" (UniqueName: \"kubernetes.io/projected/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-kube-api-access-8v8z6\") pod \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") "
Oct 30 19:38:25.383889 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.383812    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-workload-socket" pod "e8b1d71f-87d1-4743-b192-ef5219a8c0f2" (UID: "e8b1d71f-87d1-4743-b192-ef5219a8c0f2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:25.383889 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.383857    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-workload-certs\") pod \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") "
Oct 30 19:38:25.384104 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.383941    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-credential-socket" pod "e8b1d71f-87d1-4743-b192-ef5219a8c0f2" (UID: "e8b1d71f-87d1-4743-b192-ef5219a8c0f2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:25.384104 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.384008    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istiod-ca-cert" pod "e8b1d71f-87d1-4743-b192-ef5219a8c0f2" (UID: "e8b1d71f-87d1-4743-b192-ef5219a8c0f2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:38:25.384104 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.384057    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-data\") pod \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") "
Oct 30 19:38:25.384312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.384104    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-podinfo\") pod \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\" (UID: \"e8b1d71f-87d1-4743-b192-ef5219a8c0f2\") "
Oct 30 19:38:25.384312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.384269    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-workload-certs" pod "e8b1d71f-87d1-4743-b192-ef5219a8c0f2" (UID: "e8b1d71f-87d1-4743-b192-ef5219a8c0f2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:25.384432 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.384402    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-envoy\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.384520 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.384489    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-data" pod "e8b1d71f-87d1-4743-b192-ef5219a8c0f2" (UID: "e8b1d71f-87d1-4743-b192-ef5219a8c0f2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:25.384586 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.384575    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbdvr\" (UniqueName: \"kubernetes.io/projected/29b2b2ca-a677-4d48-ab5c-9b202e007103-kube-api-access-sbdvr\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.384764 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.384745    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-credential-socket\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.384843 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.384779    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-token\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.384933 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.384894    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-data\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.385083 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.385055    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-workload-certs\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.385155 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.385109    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-workload-socket\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.385155 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.385141    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-podinfo\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.385293 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.385275    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29b2b2ca-a677-4d48-ab5c-9b202e007103-istiod-ca-cert\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.385469 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.385449    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:38:25.385534 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.385475    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:38:25.385534 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.385491    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:38:25.385534 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.385505    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:38:25.385534 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.385519    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:38:25.386425 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.386401    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-token" pod "e8b1d71f-87d1-4743-b192-ef5219a8c0f2" (UID: "e8b1d71f-87d1-4743-b192-ef5219a8c0f2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:25.386562 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.386540    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-kube-api-access-8v8z6" pod "e8b1d71f-87d1-4743-b192-ef5219a8c0f2" (UID: "e8b1d71f-87d1-4743-b192-ef5219a8c0f2"). InnerVolumeSpecName "kube-api-access-8v8z6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:38:25.386759 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.386744    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-envoy" pod "e8b1d71f-87d1-4743-b192-ef5219a8c0f2" (UID: "e8b1d71f-87d1-4743-b192-ef5219a8c0f2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:38:25.386843 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.386831    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-podinfo" pod "e8b1d71f-87d1-4743-b192-ef5219a8c0f2" (UID: "e8b1d71f-87d1-4743-b192-ef5219a8c0f2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:38:25.487336 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.487294    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-credential-socket\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.487524 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.487348    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-token\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.487524 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.487461    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-data\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.487524 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.487518    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-workload-certs\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.487856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.487550    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-workload-socket\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.487856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.487578    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-podinfo\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.487856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.487658    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29b2b2ca-a677-4d48-ab5c-9b202e007103-istiod-ca-cert\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.487856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.487726    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-envoy\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.488245 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.487868    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbdvr\" (UniqueName: \"kubernetes.io/projected/29b2b2ca-a677-4d48-ab5c-9b202e007103-kube-api-access-sbdvr\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.488245 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.487988    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:38:25.488245 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.488006    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:38:25.488245 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.488022    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:38:25.488245 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.488037    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8v8z6\" (UniqueName: \"kubernetes.io/projected/e8b1d71f-87d1-4743-b192-ef5219a8c0f2-kube-api-access-8v8z6\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:38:25.488588 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.488305    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-data\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.488588 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.488481    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-workload-certs\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.489049 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.489024    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-workload-socket\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.490933 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.489969    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-credential-socket\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.490933 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.490892    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29b2b2ca-a677-4d48-ab5c-9b202e007103-istiod-ca-cert\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.491817 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.491769    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-podinfo\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.492049 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.492014    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-envoy\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.500580 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.500554    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-token\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.500753 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.500736    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbdvr\" (UniqueName: \"kubernetes.io/projected/29b2b2ca-a677-4d48-ab5c-9b202e007103-kube-api-access-sbdvr\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:25.619194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:25.616672    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:26.051392 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:26.047145    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"]
Oct 30 19:38:26.053757 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:38:26.053720    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29b2b2ca_a677_4d48_ab5c_9b202e007103.slice/crio-c7cd65618c13c7eae6e5ed6128fb8a2e9d8b615177c0a4059757a862a6b96487 WatchSource:0}: Error finding container c7cd65618c13c7eae6e5ed6128fb8a2e9d8b615177c0a4059757a862a6b96487: Status 404 returned error can't find the container with id c7cd65618c13c7eae6e5ed6128fb8a2e9d8b615177c0a4059757a862a6b96487
Oct 30 19:38:26.077212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:26.077137    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:38:26.077513 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:26.077487    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:38:26.077837 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:26.077811    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:38:26.093587 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:26.092788    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt" event={"ID":"29b2b2ca-a677-4d48-ab5c-9b202e007103","Type":"ContainerStarted","Data":"c7cd65618c13c7eae6e5ed6128fb8a2e9d8b615177c0a4059757a862a6b96487"}
Oct 30 19:38:26.101161 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:26.101090    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm" event={"ID":"e8b1d71f-87d1-4743-b192-ef5219a8c0f2","Type":"ContainerDied","Data":"9cdb1deaae8a2aa2b474e8a04fe8e1127fdac434418078f05ad2497afe39025b"}
Oct 30 19:38:26.101330 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:26.101238    4524 scope.go:122] "RemoveContainer" containerID="fc5524f96c965f1afecabd31721a287f29e943a3493ef65bc83cc222879ccb7e"
Oct 30 19:38:26.101537 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:26.101480    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"
Oct 30 19:38:26.171420 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:26.171384    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"]
Oct 30 19:38:26.191190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:26.187198    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-76117c-openshift-default-5bf558ddbc-gvckm"]
Oct 30 19:38:26.401614 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:26.401574    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e8b1d71f-87d1-4743-b192-ef5219a8c0f2" path="/var/lib/kubelet/pods/e8b1d71f-87d1-4743-b192-ef5219a8c0f2/volumes"
Oct 30 19:38:27.115835 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:27.115795    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt" event={"ID":"29b2b2ca-a677-4d48-ab5c-9b202e007103","Type":"ContainerStarted","Data":"454e70ec4c1b35565716d90254df05b5e7048c6c94ad9f417f14bb24aaf28837"}
Oct 30 19:38:27.617621 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:27.617583    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:27.618676 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:27.618637    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:38:27.618833 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:27.618697    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt" podUID="29b2b2ca-a677-4d48-ab5c-9b202e007103" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused"
Oct 30 19:38:28.621193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:28.620379    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:38:28.621193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:28.620458    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt" podUID="29b2b2ca-a677-4d48-ab5c-9b202e007103" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused"
Oct 30 19:38:29.622668 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:29.618510    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:38:29.622668 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:29.618588    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt" podUID="29b2b2ca-a677-4d48-ab5c-9b202e007103" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused"
Oct 30 19:38:30.618306 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:30.618273    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:38:30.618521 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:30.618325    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt" podUID="29b2b2ca-a677-4d48-ab5c-9b202e007103" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused"
Oct 30 19:38:31.618276 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:31.618240    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:38:31.618693 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:31.618304    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt" podUID="29b2b2ca-a677-4d48-ab5c-9b202e007103" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused"
Oct 30 19:38:32.678199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:32.674461    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused" start-of-body=
Oct 30 19:38:32.678199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:32.674530    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt" podUID="29b2b2ca-a677-4d48-ab5c-9b202e007103" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.162:15021/healthz/ready\": dial tcp 10.133.0.162:15021: connect: connection refused"
Oct 30 19:38:33.623360 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:33.623330    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:33.623875 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:33.623854    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:33.624505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:33.624483    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:38:33.653758 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:38:33.653715    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt" podStartSLOduration=33.65370367 podStartE2EDuration="33.65370367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:27.147431407 +0000 UTC m=+1938.888422483" watchObservedRunningTime="2026-10-30 19:38:33.65370367 +0000 UTC m=+1945.394694748"
Oct 30 19:38:45.627320 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:38:45.627283    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:39:15.628254 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:39:15.628215    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:39:45.630205 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:39:45.630157    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:39:49.497276 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:49.497240    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5"]
Oct 30 19:39:49.503450 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:49.503428    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5"
Oct 30 19:39:49.513223 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:49.513137    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 19:39:49.516066 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:49.516046    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z26xs\""
Oct 30 19:39:49.524081 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:49.524056    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5"]
Oct 30 19:39:49.615606 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:49.615565    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/22b2768d-5577-40ca-b8da-50fea4f50651-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b5ecb2fa-6w9g5\" (UID: \"22b2768d-5577-40ca-b8da-50fea4f50651\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5"
Oct 30 19:39:49.615782 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:49.615624    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/22b2768d-5577-40ca-b8da-50fea4f50651-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b5ecb2fa-6w9g5\" (UID: \"22b2768d-5577-40ca-b8da-50fea4f50651\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5"
Oct 30 19:39:49.615934 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:49.615906    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9j2z\" (UniqueName: \"kubernetes.io/projected/22b2768d-5577-40ca-b8da-50fea4f50651-kube-api-access-s9j2z\") pod \"maas-api-revoke-sub-b5ecb2fa-6w9g5\" (UID: \"22b2768d-5577-40ca-b8da-50fea4f50651\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5"
Oct 30 19:39:49.717616 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:49.717583    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/22b2768d-5577-40ca-b8da-50fea4f50651-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b5ecb2fa-6w9g5\" (UID: \"22b2768d-5577-40ca-b8da-50fea4f50651\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5"
Oct 30 19:39:49.717780 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:49.717644    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/22b2768d-5577-40ca-b8da-50fea4f50651-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b5ecb2fa-6w9g5\" (UID: \"22b2768d-5577-40ca-b8da-50fea4f50651\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5"
Oct 30 19:39:49.717780 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:49.717774    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9j2z\" (UniqueName: \"kubernetes.io/projected/22b2768d-5577-40ca-b8da-50fea4f50651-kube-api-access-s9j2z\") pod \"maas-api-revoke-sub-b5ecb2fa-6w9g5\" (UID: \"22b2768d-5577-40ca-b8da-50fea4f50651\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5"
Oct 30 19:39:49.718574 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:49.718546    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/22b2768d-5577-40ca-b8da-50fea4f50651-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b5ecb2fa-6w9g5\" (UID: \"22b2768d-5577-40ca-b8da-50fea4f50651\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5"
Oct 30 19:39:49.720047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:49.720020    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/22b2768d-5577-40ca-b8da-50fea4f50651-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b5ecb2fa-6w9g5\" (UID: \"22b2768d-5577-40ca-b8da-50fea4f50651\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5"
Oct 30 19:39:49.729531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:49.729509    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9j2z\" (UniqueName: \"kubernetes.io/projected/22b2768d-5577-40ca-b8da-50fea4f50651-kube-api-access-s9j2z\") pod \"maas-api-revoke-sub-b5ecb2fa-6w9g5\" (UID: \"22b2768d-5577-40ca-b8da-50fea4f50651\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5"
Oct 30 19:39:49.825250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:49.821704    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5"
Oct 30 19:39:50.181215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:50.181145    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5"]
Oct 30 19:39:50.186994 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:39:50.186970    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22b2768d_5577_40ca_b8da_50fea4f50651.slice/crio-3810c6d5f381bda94affa1fde3a4bbdd3b7ec42469f7c94dea02e6ed33cb3863 WatchSource:0}: Error finding container 3810c6d5f381bda94affa1fde3a4bbdd3b7ec42469f7c94dea02e6ed33cb3863: Status 404 returned error can't find the container with id 3810c6d5f381bda94affa1fde3a4bbdd3b7ec42469f7c94dea02e6ed33cb3863
Oct 30 19:39:50.749724 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:50.749692    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5" event={"ID":"22b2768d-5577-40ca-b8da-50fea4f50651","Type":"ContainerStarted","Data":"3810c6d5f381bda94affa1fde3a4bbdd3b7ec42469f7c94dea02e6ed33cb3863"}
Oct 30 19:39:51.760304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:51.760164    4524 generic.go:356] "Generic (PLEG): container finished" podID="22b2768d-5577-40ca-b8da-50fea4f50651" containerID="684f1dde6618fa1403bfd02c563a31d343c0b53805db59de4e7719241b53c13d" exitCode=0
Oct 30 19:39:51.760304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:51.760234    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5" event={"ID":"22b2768d-5577-40ca-b8da-50fea4f50651","Type":"ContainerDied","Data":"684f1dde6618fa1403bfd02c563a31d343c0b53805db59de4e7719241b53c13d"}
Oct 30 19:39:52.078765 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:52.078652    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"]
Oct 30 19:39:52.078931 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:52.078847    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4" podUID="3b269ee8-7299-4e79-8f77-ed5e4e76730d" containerName="istio-proxy" containerID="cri-o://dc37e825ba702110e87ec82948eddc602349e7d442cc1dd540e1a126046d1d6f" gracePeriod=30
Oct 30 19:39:52.995727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:52.995697    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5"
Oct 30 19:39:53.102769 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:53.102735    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/22b2768d-5577-40ca-b8da-50fea4f50651-kube-api-access-s9j2z\" (UniqueName: \"kubernetes.io/projected/22b2768d-5577-40ca-b8da-50fea4f50651-kube-api-access-s9j2z\") pod \"22b2768d-5577-40ca-b8da-50fea4f50651\" (UID: \"22b2768d-5577-40ca-b8da-50fea4f50651\") "
Oct 30 19:39:53.103146 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:53.103115    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/22b2768d-5577-40ca-b8da-50fea4f50651-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/22b2768d-5577-40ca-b8da-50fea4f50651-maas-api-openshift-ca\") pod \"22b2768d-5577-40ca-b8da-50fea4f50651\" (UID: \"22b2768d-5577-40ca-b8da-50fea4f50651\") "
Oct 30 19:39:53.103450 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:53.103427    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/22b2768d-5577-40ca-b8da-50fea4f50651-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/22b2768d-5577-40ca-b8da-50fea4f50651-maas-api-xks-ca\") pod \"22b2768d-5577-40ca-b8da-50fea4f50651\" (UID: \"22b2768d-5577-40ca-b8da-50fea4f50651\") "
Oct 30 19:39:53.104025 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:53.103985    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22b2768d-5577-40ca-b8da-50fea4f50651-maas-api-openshift-ca" pod "22b2768d-5577-40ca-b8da-50fea4f50651" (UID: "22b2768d-5577-40ca-b8da-50fea4f50651"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:39:53.104233 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:53.104212    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/22b2768d-5577-40ca-b8da-50fea4f50651-maas-api-openshift-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:39:53.112212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:53.108453    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22b2768d-5577-40ca-b8da-50fea4f50651-maas-api-xks-ca" pod "22b2768d-5577-40ca-b8da-50fea4f50651" (UID: "22b2768d-5577-40ca-b8da-50fea4f50651"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:39:53.112212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:53.108674    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22b2768d-5577-40ca-b8da-50fea4f50651-kube-api-access-s9j2z" pod "22b2768d-5577-40ca-b8da-50fea4f50651" (UID: "22b2768d-5577-40ca-b8da-50fea4f50651"). InnerVolumeSpecName "kube-api-access-s9j2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:39:53.205538 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:53.205508    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9j2z\" (UniqueName: \"kubernetes.io/projected/22b2768d-5577-40ca-b8da-50fea4f50651-kube-api-access-s9j2z\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:39:53.205538 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:53.205536    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/22b2768d-5577-40ca-b8da-50fea4f50651-maas-api-xks-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:39:53.795722 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:53.795681    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5" event={"ID":"22b2768d-5577-40ca-b8da-50fea4f50651","Type":"ContainerDied","Data":"3810c6d5f381bda94affa1fde3a4bbdd3b7ec42469f7c94dea02e6ed33cb3863"}
Oct 30 19:39:53.795950 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:53.795755    4524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3810c6d5f381bda94affa1fde3a4bbdd3b7ec42469f7c94dea02e6ed33cb3863"
Oct 30 19:39:53.795950 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:53.795853    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5"
Oct 30 19:39:57.451296 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.450210    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:39:57.522834 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.522795    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n"]
Oct 30 19:39:57.524278 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.524238    4524 cpu_manager.go:414] "containerMap: removing container" podUID="22b2768d-5577-40ca-b8da-50fea4f50651" containerName="revoke-keys"
Oct 30 19:39:57.524436 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.524422    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="22b2768d-5577-40ca-b8da-50fea4f50651" containerName="revoke-keys"
Oct 30 19:39:57.524592 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.524581    4524 cpu_manager.go:414] "containerMap: removing container" podUID="3b269ee8-7299-4e79-8f77-ed5e4e76730d" containerName="istio-proxy"
Oct 30 19:39:57.524710 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.524699    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b269ee8-7299-4e79-8f77-ed5e4e76730d" containerName="istio-proxy"
Oct 30 19:39:57.536316 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.533224    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b269ee8-7299-4e79-8f77-ed5e4e76730d" containerName="istio-proxy"
Oct 30 19:39:57.536316 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.533263    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="22b2768d-5577-40ca-b8da-50fea4f50651" containerName="revoke-keys"
Oct 30 19:39:57.540091 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.540037    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n"]
Oct 30 19:39:57.540271 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.540147    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n"
Oct 30 19:39:57.548725 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.548683    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-7e6199\""
Oct 30 19:39:57.549838 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.549737    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-7e6199-dockercfg-xrq29\""
Oct 30 19:39:57.557580 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.556775    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-credential-socket\") pod \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") "
Oct 30 19:39:57.557580 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.556819    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-workload-socket\") pod \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") "
Oct 30 19:39:57.557580 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.557018    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-workload-certs\") pod \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") "
Oct 30 19:39:57.557580 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.557116    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-podinfo\") pod \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") "
Oct 30 19:39:57.557580 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.557207    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-token\" (UniqueName: \"kubernetes.io/projected/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-token\") pod \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") "
Oct 30 19:39:57.557580 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.557268    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-credential-socket" pod "3b269ee8-7299-4e79-8f77-ed5e4e76730d" (UID: "3b269ee8-7299-4e79-8f77-ed5e4e76730d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:57.557580 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.557289    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istiod-ca-cert\") pod \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") "
Oct 30 19:39:57.557580 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.557329    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b269ee8-7299-4e79-8f77-ed5e4e76730d-kube-api-access-5q8t8\" (UniqueName: \"kubernetes.io/projected/3b269ee8-7299-4e79-8f77-ed5e4e76730d-kube-api-access-5q8t8\") pod \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") "
Oct 30 19:39:57.557580 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.557371    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-data\") pod \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") "
Oct 30 19:39:57.557580 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.557482    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-envoy\") pod \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\" (UID: \"3b269ee8-7299-4e79-8f77-ed5e4e76730d\") "
Oct 30 19:39:57.558268 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.557824    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-workload-certs" pod "3b269ee8-7299-4e79-8f77-ed5e4e76730d" (UID: "3b269ee8-7299-4e79-8f77-ed5e4e76730d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:57.558336 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.558314    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/017eaed3-ec30-4831-b3eb-86ac16a340af-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n\" (UID: \"017eaed3-ec30-4831-b3eb-86ac16a340af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n"
Oct 30 19:39:57.558420 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.558398    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istiod-ca-cert" pod "3b269ee8-7299-4e79-8f77-ed5e4e76730d" (UID: "3b269ee8-7299-4e79-8f77-ed5e4e76730d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:39:57.558494 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.558476    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dn92\" (UniqueName: \"kubernetes.io/projected/017eaed3-ec30-4831-b3eb-86ac16a340af-kube-api-access-7dn92\") pod \"payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n\" (UID: \"017eaed3-ec30-4831-b3eb-86ac16a340af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n"
Oct 30 19:39:57.558942 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.558917    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-workload-socket" pod "3b269ee8-7299-4e79-8f77-ed5e4e76730d" (UID: "3b269ee8-7299-4e79-8f77-ed5e4e76730d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:57.559708 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.559687    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-data" pod "3b269ee8-7299-4e79-8f77-ed5e4e76730d" (UID: "3b269ee8-7299-4e79-8f77-ed5e4e76730d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:57.560015 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.559893    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:39:57.560015 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.559923    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:39:57.560015 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.559937    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:39:57.560850 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.560797    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-podinfo" pod "3b269ee8-7299-4e79-8f77-ed5e4e76730d" (UID: "3b269ee8-7299-4e79-8f77-ed5e4e76730d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:39:57.563706 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.563266    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b269ee8-7299-4e79-8f77-ed5e4e76730d-kube-api-access-5q8t8" pod "3b269ee8-7299-4e79-8f77-ed5e4e76730d" (UID: "3b269ee8-7299-4e79-8f77-ed5e4e76730d"). InnerVolumeSpecName "kube-api-access-5q8t8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:39:57.564747 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.564506    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-envoy" pod "3b269ee8-7299-4e79-8f77-ed5e4e76730d" (UID: "3b269ee8-7299-4e79-8f77-ed5e4e76730d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:39:57.564882 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.564858    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-token" pod "3b269ee8-7299-4e79-8f77-ed5e4e76730d" (UID: "3b269ee8-7299-4e79-8f77-ed5e4e76730d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:39:57.660934 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.660864    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/017eaed3-ec30-4831-b3eb-86ac16a340af-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n\" (UID: \"017eaed3-ec30-4831-b3eb-86ac16a340af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n"
Oct 30 19:39:57.661084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.660944    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dn92\" (UniqueName: \"kubernetes.io/projected/017eaed3-ec30-4831-b3eb-86ac16a340af-kube-api-access-7dn92\") pod \"payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n\" (UID: \"017eaed3-ec30-4831-b3eb-86ac16a340af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n"
Oct 30 19:39:57.661157 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.661142    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:39:57.661256 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.661162    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:39:57.661256 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.661224    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5q8t8\" (UniqueName: \"kubernetes.io/projected/3b269ee8-7299-4e79-8f77-ed5e4e76730d-kube-api-access-5q8t8\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:39:57.661256 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.661234    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:39:57.661256 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.661243    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:39:57.661256 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.661252    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b269ee8-7299-4e79-8f77-ed5e4e76730d-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:39:57.661619 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.661597    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/017eaed3-ec30-4831-b3eb-86ac16a340af-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n\" (UID: \"017eaed3-ec30-4831-b3eb-86ac16a340af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n"
Oct 30 19:39:57.675080 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.675057    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dn92\" (UniqueName: \"kubernetes.io/projected/017eaed3-ec30-4831-b3eb-86ac16a340af-kube-api-access-7dn92\") pod \"payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n\" (UID: \"017eaed3-ec30-4831-b3eb-86ac16a340af\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n"
Oct 30 19:39:57.841487 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.841451    4524 generic.go:356] "Generic (PLEG): container finished" podID="3b269ee8-7299-4e79-8f77-ed5e4e76730d" containerID="dc37e825ba702110e87ec82948eddc602349e7d442cc1dd540e1a126046d1d6f" exitCode=0
Oct 30 19:39:57.841639 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.841500    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"
Oct 30 19:39:57.841639 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.841531    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4" event={"ID":"3b269ee8-7299-4e79-8f77-ed5e4e76730d","Type":"ContainerDied","Data":"dc37e825ba702110e87ec82948eddc602349e7d442cc1dd540e1a126046d1d6f"}
Oct 30 19:39:57.841639 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.841585    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4" event={"ID":"3b269ee8-7299-4e79-8f77-ed5e4e76730d","Type":"ContainerDied","Data":"f3b3eeb14592ccbb469605c9ceed609858eae0537993b658e5c9bd509742dc97"}
Oct 30 19:39:57.841639 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.841606    4524 scope.go:122] "RemoveContainer" containerID="dc37e825ba702110e87ec82948eddc602349e7d442cc1dd540e1a126046d1d6f"
Oct 30 19:39:57.853010 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.852990    4524 scope.go:122] "RemoveContainer" containerID="dc37e825ba702110e87ec82948eddc602349e7d442cc1dd540e1a126046d1d6f"
Oct 30 19:39:57.853305 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:39:57.853286    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc37e825ba702110e87ec82948eddc602349e7d442cc1dd540e1a126046d1d6f\": container with ID starting with dc37e825ba702110e87ec82948eddc602349e7d442cc1dd540e1a126046d1d6f not found: ID does not exist" containerID="dc37e825ba702110e87ec82948eddc602349e7d442cc1dd540e1a126046d1d6f"
Oct 30 19:39:57.853381 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.853312    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc37e825ba702110e87ec82948eddc602349e7d442cc1dd540e1a126046d1d6f"} err="failed to get container status \"dc37e825ba702110e87ec82948eddc602349e7d442cc1dd540e1a126046d1d6f\": rpc error: code = NotFound desc = could not find container \"dc37e825ba702110e87ec82948eddc602349e7d442cc1dd540e1a126046d1d6f\": container with ID starting with dc37e825ba702110e87ec82948eddc602349e7d442cc1dd540e1a126046d1d6f not found: ID does not exist"
Oct 30 19:39:57.867909 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.867841    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"]
Oct 30 19:39:57.875246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.875222    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-bktq4"]
Oct 30 19:39:57.924485 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:57.924411    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n"
Oct 30 19:39:58.309269 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:58.309238    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n"]
Oct 30 19:39:58.309742 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:39:58.309712    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod017eaed3_ec30_4831_b3eb_86ac16a340af.slice/crio-1ae87fc727e1c874f1241d37a8faa7e62da2d011a470927947b2b8db78605946 WatchSource:0}: Error finding container 1ae87fc727e1c874f1241d37a8faa7e62da2d011a470927947b2b8db78605946: Status 404 returned error can't find the container with id 1ae87fc727e1c874f1241d37a8faa7e62da2d011a470927947b2b8db78605946
Oct 30 19:39:58.397359 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:58.397331    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3b269ee8-7299-4e79-8f77-ed5e4e76730d" path="/var/lib/kubelet/pods/3b269ee8-7299-4e79-8f77-ed5e4e76730d/volumes"
Oct 30 19:39:58.852019 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:58.851977    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n" event={"ID":"017eaed3-ec30-4831-b3eb-86ac16a340af","Type":"ContainerStarted","Data":"0d5ef1781479ecb805b240150da8ca40eed300d79e8c17caca42af1e54a685fe"}
Oct 30 19:39:58.852499 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:58.852044    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n"
Oct 30 19:39:58.852499 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:39:58.852063    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n" event={"ID":"017eaed3-ec30-4831-b3eb-86ac16a340af","Type":"ContainerStarted","Data":"1ae87fc727e1c874f1241d37a8faa7e62da2d011a470927947b2b8db78605946"}
Oct 30 19:40:00.260059 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.259991    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n" podStartSLOduration=5.259972236 podStartE2EDuration="5.259972236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:58.871800387 +0000 UTC m=+2030.612791460" watchObservedRunningTime="2026-10-30 19:40:00.259972236 +0000 UTC m=+2032.000963318"
Oct 30 19:40:00.265634 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.265604    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"]
Oct 30 19:40:00.266351 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.266323    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s" podUID="6eecd016-5523-46a5-919b-09e0772cba31" containerName="maas-api" containerID="cri-o://43ee0bd9ff2db32c4b7b84b353b49acb65927a51351b145138561700132edb6e" gracePeriod=30
Oct 30 19:40:00.797069 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.796649    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"
Oct 30 19:40:00.861018 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.860985    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"]
Oct 30 19:40:00.862581 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.862555    4524 cpu_manager.go:414] "containerMap: removing container" podUID="6eecd016-5523-46a5-919b-09e0772cba31" containerName="maas-api"
Oct 30 19:40:00.862729 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.862717    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6eecd016-5523-46a5-919b-09e0772cba31" containerName="maas-api"
Oct 30 19:40:00.863068 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.863053    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="6eecd016-5523-46a5-919b-09e0772cba31" containerName="maas-api"
Oct 30 19:40:00.880522 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.879769    4524 generic.go:356] "Generic (PLEG): container finished" podID="6eecd016-5523-46a5-919b-09e0772cba31" containerID="43ee0bd9ff2db32c4b7b84b353b49acb65927a51351b145138561700132edb6e" exitCode=0
Oct 30 19:40:00.880522 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.879991    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"
Oct 30 19:40:00.924335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.922141    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6eecd016-5523-46a5-919b-09e0772cba31-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eecd016-5523-46a5-919b-09e0772cba31-metrics-certs\") pod \"6eecd016-5523-46a5-919b-09e0772cba31\" (UID: \"6eecd016-5523-46a5-919b-09e0772cba31\") "
Oct 30 19:40:00.924335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.922235    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6eecd016-5523-46a5-919b-09e0772cba31-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6eecd016-5523-46a5-919b-09e0772cba31-openshift-service-ca\") pod \"6eecd016-5523-46a5-919b-09e0772cba31\" (UID: \"6eecd016-5523-46a5-919b-09e0772cba31\") "
Oct 30 19:40:00.924335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.922341    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6eecd016-5523-46a5-919b-09e0772cba31-kube-api-access-l728v\" (UniqueName: \"kubernetes.io/projected/6eecd016-5523-46a5-919b-09e0772cba31-kube-api-access-l728v\") pod \"6eecd016-5523-46a5-919b-09e0772cba31\" (UID: \"6eecd016-5523-46a5-919b-09e0772cba31\") "
Oct 30 19:40:00.924335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.922380    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6eecd016-5523-46a5-919b-09e0772cba31-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6eecd016-5523-46a5-919b-09e0772cba31-maas-api-tls\") pod \"6eecd016-5523-46a5-919b-09e0772cba31\" (UID: \"6eecd016-5523-46a5-919b-09e0772cba31\") "
Oct 30 19:40:00.924335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.924035    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6eecd016-5523-46a5-919b-09e0772cba31-openshift-service-ca" pod "6eecd016-5523-46a5-919b-09e0772cba31" (UID: "6eecd016-5523-46a5-919b-09e0772cba31"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:40:00.932640 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.932579    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s" event={"ID":"6eecd016-5523-46a5-919b-09e0772cba31","Type":"ContainerDied","Data":"43ee0bd9ff2db32c4b7b84b353b49acb65927a51351b145138561700132edb6e"}
Oct 30 19:40:00.932804 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.932645    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s" event={"ID":"6eecd016-5523-46a5-919b-09e0772cba31","Type":"ContainerDied","Data":"de6a3ff2e048f2764264243be91a7eba54b26a444d76426f62f9dda5378c07b6"}
Oct 30 19:40:00.932804 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.932680    4524 scope.go:122] "RemoveContainer" containerID="43ee0bd9ff2db32c4b7b84b353b49acb65927a51351b145138561700132edb6e"
Oct 30 19:40:00.933138 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.932908    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:00.946754 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.939032    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w0-2405fb-openshift-default-dockercfg-mpx47\""
Oct 30 19:40:00.946754 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.942346    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6eecd016-5523-46a5-919b-09e0772cba31-metrics-certs" pod "6eecd016-5523-46a5-919b-09e0772cba31" (UID: "6eecd016-5523-46a5-919b-09e0772cba31"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:40:00.946754 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.945276    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6eecd016-5523-46a5-919b-09e0772cba31-maas-api-tls" pod "6eecd016-5523-46a5-919b-09e0772cba31" (UID: "6eecd016-5523-46a5-919b-09e0772cba31"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:40:00.997243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:00.980428    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6eecd016-5523-46a5-919b-09e0772cba31-kube-api-access-l728v" pod "6eecd016-5523-46a5-919b-09e0772cba31" (UID: "6eecd016-5523-46a5-919b-09e0772cba31"). InnerVolumeSpecName "kube-api-access-l728v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:01.002927 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.002883    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"]
Oct 30 19:40:01.065956 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.041304    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-workload-socket\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.065956 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.041385    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8htzp\" (UniqueName: \"kubernetes.io/projected/0ad9005d-b179-48d1-8a9f-67fff6724a6f-kube-api-access-8htzp\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.065956 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.041427    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-data\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.065956 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.041500    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-podinfo\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.065956 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.041543    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istiod-ca-cert\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.065956 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.041587    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-workload-certs\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.065956 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.041621    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-token\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.065956 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.041710    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-credential-socket\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.065956 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.041752    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-envoy\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.065956 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.041903    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l728v\" (UniqueName: \"kubernetes.io/projected/6eecd016-5523-46a5-919b-09e0772cba31-kube-api-access-l728v\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:01.065956 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.041922    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6eecd016-5523-46a5-919b-09e0772cba31-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:01.065956 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.041936    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6eecd016-5523-46a5-919b-09e0772cba31-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:01.065956 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.041948    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6eecd016-5523-46a5-919b-09e0772cba31-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:01.111977 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.111944    4524 scope.go:122] "RemoveContainer" containerID="43ee0bd9ff2db32c4b7b84b353b49acb65927a51351b145138561700132edb6e"
Oct 30 19:40:01.112902 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:40:01.112869    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43ee0bd9ff2db32c4b7b84b353b49acb65927a51351b145138561700132edb6e\": container with ID starting with 43ee0bd9ff2db32c4b7b84b353b49acb65927a51351b145138561700132edb6e not found: ID does not exist" containerID="43ee0bd9ff2db32c4b7b84b353b49acb65927a51351b145138561700132edb6e"
Oct 30 19:40:01.113098 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.112918    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43ee0bd9ff2db32c4b7b84b353b49acb65927a51351b145138561700132edb6e"} err="failed to get container status \"43ee0bd9ff2db32c4b7b84b353b49acb65927a51351b145138561700132edb6e\": rpc error: code = NotFound desc = could not find container \"43ee0bd9ff2db32c4b7b84b353b49acb65927a51351b145138561700132edb6e\": container with ID starting with 43ee0bd9ff2db32c4b7b84b353b49acb65927a51351b145138561700132edb6e not found: ID does not exist"
Oct 30 19:40:01.143131 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.143100    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-workload-certs\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.143283 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.143140    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-token\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.143283 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.143239    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-credential-socket\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.143418 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.143281    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-envoy\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.143515 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.143494    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-workload-socket\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.143580 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.143564    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8htzp\" (UniqueName: \"kubernetes.io/projected/0ad9005d-b179-48d1-8a9f-67fff6724a6f-kube-api-access-8htzp\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.143636 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.143600    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-data\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.143695 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.143675    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-podinfo\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.143757 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.143704    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-credential-socket\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.143757 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.143720    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istiod-ca-cert\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.143924 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.143503    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-workload-certs\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.144094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.144069    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-workload-socket\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.144181 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.144126    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-data\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.144426 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.144407    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istiod-ca-cert\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.145788 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.145766    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-envoy\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.146591 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.146570    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-podinfo\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.159728 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.159704    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-token\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.159951 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.159933    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8htzp\" (UniqueName: \"kubernetes.io/projected/0ad9005d-b179-48d1-8a9f-67fff6724a6f-kube-api-access-8htzp\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.209422 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.209395    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"]
Oct 30 19:40:01.220857 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.220826    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1fa500-d4b7f877f-dqf7s"]
Oct 30 19:40:01.314642 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.314616    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:01.617766 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.617731    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"]
Oct 30 19:40:01.620319 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.620274    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:40:01.620437 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.620379    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:40:01.620437 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.620422    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:40:01.963246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.957018    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv" event={"ID":"0ad9005d-b179-48d1-8a9f-67fff6724a6f","Type":"ContainerStarted","Data":"ff4d2730b5a342a1814c559e5c7442f882394ca42b8825dada709b95477ca051"}
Oct 30 19:40:01.963246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.957107    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv" event={"ID":"0ad9005d-b179-48d1-8a9f-67fff6724a6f","Type":"ContainerStarted","Data":"ffaf98a9347e8d99ad0ebdbada4c9d87a92ff962a5ef2759fc57d0faa9ba4b22"}
Oct 30 19:40:01.996472 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:01.996414    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv" podStartSLOduration=3.996396516 podStartE2EDuration="3.996396516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:01.995439993 +0000 UTC m=+2033.736431073" watchObservedRunningTime="2026-10-30 19:40:01.996396516 +0000 UTC m=+2033.737387594"
Oct 30 19:40:02.315048 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:02.314964    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:02.316748 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:02.316722    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:40:02.316861 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:02.316774    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv" podUID="0ad9005d-b179-48d1-8a9f-67fff6724a6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused"
Oct 30 19:40:02.406254 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:02.406223    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6eecd016-5523-46a5-919b-09e0772cba31" path="/var/lib/kubelet/pods/6eecd016-5523-46a5-919b-09e0772cba31/volumes"
Oct 30 19:40:03.316184 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:03.316119    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:40:03.317226 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:03.317183    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv" podUID="0ad9005d-b179-48d1-8a9f-67fff6724a6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused"
Oct 30 19:40:04.315347 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:04.315303    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:40:04.315516 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:04.315369    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv" podUID="0ad9005d-b179-48d1-8a9f-67fff6724a6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused"
Oct 30 19:40:05.315419 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:05.315380    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:40:05.315934 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:05.315448    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv" podUID="0ad9005d-b179-48d1-8a9f-67fff6724a6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused"
Oct 30 19:40:06.315073 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:06.315040    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:40:06.315274 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:06.315091    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv" podUID="0ad9005d-b179-48d1-8a9f-67fff6724a6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused"
Oct 30 19:40:07.057467 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:07.057433    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"]
Oct 30 19:40:07.058408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:07.058377    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr" podUID="b0cd7cfe-e360-45e7-a17c-5067167d3afe" containerName="istio-proxy" containerID="cri-o://c8cee943882a26ddc3a76db9577794b618c92395baa26fc87529fe57faeffa99" gracePeriod=30
Oct 30 19:40:07.315842 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:07.315742    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused" start-of-body=
Oct 30 19:40:07.315842 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:07.315795    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv" podUID="0ad9005d-b179-48d1-8a9f-67fff6724a6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.165:15021/healthz/ready\": dial tcp 10.133.0.165:15021: connect: connection refused"
Oct 30 19:40:08.419404 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:08.419370    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:08.420610 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:08.420584    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:08.421985 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:08.421961    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:40:09.864301 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:09.864256    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n"
Oct 30 19:40:12.486725 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.486695    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:40:12.535622 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.534893    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"]
Oct 30 19:40:12.536317 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.536280    4524 cpu_manager.go:414] "containerMap: removing container" podUID="b0cd7cfe-e360-45e7-a17c-5067167d3afe" containerName="istio-proxy"
Oct 30 19:40:12.536317 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.536307    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0cd7cfe-e360-45e7-a17c-5067167d3afe" containerName="istio-proxy"
Oct 30 19:40:12.536533 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.536496    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="b0cd7cfe-e360-45e7-a17c-5067167d3afe" containerName="istio-proxy"
Oct 30 19:40:12.562829 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.562799    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"]
Oct 30 19:40:12.563137 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.562880    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.623538 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.623455    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-data\") pod \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") "
Oct 30 19:40:12.623538 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.623490    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istiod-ca-cert\") pod \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") "
Oct 30 19:40:12.623783 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.623561    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0cd7cfe-e360-45e7-a17c-5067167d3afe-kube-api-access-rmxxq\" (UniqueName: \"kubernetes.io/projected/b0cd7cfe-e360-45e7-a17c-5067167d3afe-kube-api-access-rmxxq\") pod \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") "
Oct 30 19:40:12.623783 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.623582    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-podinfo\") pod \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") "
Oct 30 19:40:12.623783 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.623614    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-token\" (UniqueName: \"kubernetes.io/projected/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-token\") pod \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") "
Oct 30 19:40:12.623783 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.623673    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-credential-socket\") pod \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") "
Oct 30 19:40:12.623783 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.623722    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-workload-certs\") pod \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") "
Oct 30 19:40:12.624079 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.623778    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-data" pod "b0cd7cfe-e360-45e7-a17c-5067167d3afe" (UID: "b0cd7cfe-e360-45e7-a17c-5067167d3afe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:12.624079 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.623816    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-workload-socket\") pod \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") "
Oct 30 19:40:12.624079 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.623900    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-envoy\") pod \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\" (UID: \"b0cd7cfe-e360-45e7-a17c-5067167d3afe\") "
Oct 30 19:40:12.624312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.624115    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-credential-socket" pod "b0cd7cfe-e360-45e7-a17c-5067167d3afe" (UID: "b0cd7cfe-e360-45e7-a17c-5067167d3afe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:12.624312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.624141    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-workload-socket" pod "b0cd7cfe-e360-45e7-a17c-5067167d3afe" (UID: "b0cd7cfe-e360-45e7-a17c-5067167d3afe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:12.624312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.624269    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-workload-certs" pod "b0cd7cfe-e360-45e7-a17c-5067167d3afe" (UID: "b0cd7cfe-e360-45e7-a17c-5067167d3afe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:12.624312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.624292    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istiod-ca-cert\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.624690 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.624431    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-credential-socket\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.624690 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.624518    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-token\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.624690 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.624586    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-workload-socket\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.624863 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.624744    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-envoy\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.625007 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.624986    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istiod-ca-cert" pod "b0cd7cfe-e360-45e7-a17c-5067167d3afe" (UID: "b0cd7cfe-e360-45e7-a17c-5067167d3afe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:40:12.625147 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.625122    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-workload-certs\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.625246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.625226    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-podinfo\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.625403 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.625383    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-data\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.625488 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.625470    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gpnh\" (UniqueName: \"kubernetes.io/projected/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-kube-api-access-5gpnh\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.625653 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.625636    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:12.625712 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.625669    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:12.625712 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.625684    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:12.625712 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.625698    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:12.625874 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.625712    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:12.626866 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.626848    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-token" pod "b0cd7cfe-e360-45e7-a17c-5067167d3afe" (UID: "b0cd7cfe-e360-45e7-a17c-5067167d3afe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:12.626991 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.626968    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0cd7cfe-e360-45e7-a17c-5067167d3afe-kube-api-access-rmxxq" pod "b0cd7cfe-e360-45e7-a17c-5067167d3afe" (UID: "b0cd7cfe-e360-45e7-a17c-5067167d3afe"). InnerVolumeSpecName "kube-api-access-rmxxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:12.627585 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.627561    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-envoy" pod "b0cd7cfe-e360-45e7-a17c-5067167d3afe" (UID: "b0cd7cfe-e360-45e7-a17c-5067167d3afe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:40:12.628370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.628348    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-podinfo" pod "b0cd7cfe-e360-45e7-a17c-5067167d3afe" (UID: "b0cd7cfe-e360-45e7-a17c-5067167d3afe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:40:12.727446 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.727413    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gpnh\" (UniqueName: \"kubernetes.io/projected/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-kube-api-access-5gpnh\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.727643 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.727533    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istiod-ca-cert\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.727643 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.727606    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-credential-socket\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.727643 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.727629    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-token\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.727845 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.727678    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-workload-socket\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.727845 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.727786    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-envoy\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.728200 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.727854    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-workload-certs\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.728200 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.727889    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-podinfo\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.728200 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.728029    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-data\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.728416 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.728207    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmxxq\" (UniqueName: \"kubernetes.io/projected/b0cd7cfe-e360-45e7-a17c-5067167d3afe-kube-api-access-rmxxq\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:12.728416 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.728228    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:12.728416 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.728241    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:12.728416 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.728257    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0cd7cfe-e360-45e7-a17c-5067167d3afe-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:12.728698 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.728616    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-data\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.729134 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.728893    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-workload-socket\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.729134 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.729033    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istiod-ca-cert\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.729134 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.729084    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-credential-socket\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.729492 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.729393    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-workload-certs\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.731701 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.731648    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-podinfo\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.731796 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.731723    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-envoy\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.739935 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.739915    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gpnh\" (UniqueName: \"kubernetes.io/projected/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-kube-api-access-5gpnh\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.740842 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.740822    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-token\") pod \"e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") " pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:12.895237 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:12.887252    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:13.187823 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:13.187740    4524 generic.go:356] "Generic (PLEG): container finished" podID="b0cd7cfe-e360-45e7-a17c-5067167d3afe" containerID="c8cee943882a26ddc3a76db9577794b618c92395baa26fc87529fe57faeffa99" exitCode=0
Oct 30 19:40:13.188062 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:13.187799    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"
Oct 30 19:40:13.188218 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:13.187815    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr" event={"ID":"b0cd7cfe-e360-45e7-a17c-5067167d3afe","Type":"ContainerDied","Data":"c8cee943882a26ddc3a76db9577794b618c92395baa26fc87529fe57faeffa99"}
Oct 30 19:40:13.188367 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:13.188343    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr" event={"ID":"b0cd7cfe-e360-45e7-a17c-5067167d3afe","Type":"ContainerDied","Data":"d0b79ffe679314c34c76ddbb0a288acb41e681c4acec4b0aaf014d6b7ba6b19a"}
Oct 30 19:40:13.188453 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:13.188384    4524 scope.go:122] "RemoveContainer" containerID="c8cee943882a26ddc3a76db9577794b618c92395baa26fc87529fe57faeffa99"
Oct 30 19:40:13.222483 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:13.222455    4524 scope.go:122] "RemoveContainer" containerID="c8cee943882a26ddc3a76db9577794b618c92395baa26fc87529fe57faeffa99"
Oct 30 19:40:13.222995 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:40:13.222952    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8cee943882a26ddc3a76db9577794b618c92395baa26fc87529fe57faeffa99\": container with ID starting with c8cee943882a26ddc3a76db9577794b618c92395baa26fc87529fe57faeffa99 not found: ID does not exist" containerID="c8cee943882a26ddc3a76db9577794b618c92395baa26fc87529fe57faeffa99"
Oct 30 19:40:13.223110 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:13.223003    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8cee943882a26ddc3a76db9577794b618c92395baa26fc87529fe57faeffa99"} err="failed to get container status \"c8cee943882a26ddc3a76db9577794b618c92395baa26fc87529fe57faeffa99\": rpc error: code = NotFound desc = could not find container \"c8cee943882a26ddc3a76db9577794b618c92395baa26fc87529fe57faeffa99\": container with ID starting with c8cee943882a26ddc3a76db9577794b618c92395baa26fc87529fe57faeffa99 not found: ID does not exist"
Oct 30 19:40:13.231808 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:13.231773    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"]
Oct 30 19:40:13.237277 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:13.237256    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-27b5d5-openshift-default-66b797556c-6mgnr"]
Oct 30 19:40:13.285285 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:40:13.285253    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf95c8550_35b7_4f3d_9b2d_b0a3d8022a7f.slice/crio-10624bba90ec243c78df544c67667efa61e6cbedcd0382a7e62edec5190e716e WatchSource:0}: Error finding container 10624bba90ec243c78df544c67667efa61e6cbedcd0382a7e62edec5190e716e: Status 404 returned error can't find the container with id 10624bba90ec243c78df544c67667efa61e6cbedcd0382a7e62edec5190e716e
Oct 30 19:40:13.289217 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:13.289160    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:40:13.289311 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:13.289268    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:40:13.289350 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:13.289307    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:40:13.290116 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:13.290085    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"]
Oct 30 19:40:14.218935 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:14.218873    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj" event={"ID":"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f","Type":"ContainerStarted","Data":"1f13892b8c237be127bbc24d152df5bd364541da9700573024572b9770672eb5"}
Oct 30 19:40:14.219373 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:14.218945    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj" event={"ID":"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f","Type":"ContainerStarted","Data":"10624bba90ec243c78df544c67667efa61e6cbedcd0382a7e62edec5190e716e"}
Oct 30 19:40:14.247779 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:14.247727    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj" podStartSLOduration=16.247713724 podStartE2EDuration="16.247713724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:14.243958353 +0000 UTC m=+2045.984949433" watchObservedRunningTime="2026-10-30 19:40:14.247713724 +0000 UTC m=+2045.988704801"
Oct 30 19:40:14.407809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:14.407763    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b0cd7cfe-e360-45e7-a17c-5067167d3afe" path="/var/lib/kubelet/pods/b0cd7cfe-e360-45e7-a17c-5067167d3afe/volumes"
Oct 30 19:40:14.895362 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:14.890806    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:14.895362 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:14.892079    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:40:14.895362 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:14.892134    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj" podUID="f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 19:40:15.629770 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:40:15.629733    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:40:15.897248 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:15.895556    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:40:15.897248 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:15.895617    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj" podUID="f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 19:40:16.892192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:16.891585    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:40:16.892192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:16.891650    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj" podUID="f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 19:40:17.892194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:17.891589    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:40:17.892194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:17.891649    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj" podUID="f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 19:40:18.895194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:18.890389    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:40:18.895194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:18.890448    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj" podUID="f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 19:40:19.887916 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:19.887878    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused" start-of-body=
Oct 30 19:40:19.888101 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:19.887942    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj" podUID="f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.166:15021/healthz/ready\": dial tcp 10.133.0.166:15021: connect: connection refused"
Oct 30 19:40:20.897947 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:20.896319    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:20.897947 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:20.897808    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:20.898687 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:20.898663    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:40:34.721873 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:34.721826    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n"]
Oct 30 19:40:39.727550 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:39.727501    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n" podUID="017eaed3-ec30-4831-b3eb-86ac16a340af" containerName="payload-processing" containerID="cri-o://0d5ef1781479ecb805b240150da8ca40eed300d79e8c17caca42af1e54a685fe" gracePeriod=25
Oct 30 19:40:39.861521 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:40:39.861498    4524 logging.go:55] [core] [Channel #5642 SubChannel #5643]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.164:50052", ServerName: "10.133.0.164:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.164:50052: connect: connection refused"
Oct 30 19:40:40.044238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.044203    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n"
Oct 30 19:40:40.113599 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.113559    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"]
Oct 30 19:40:40.117114 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.117082    4524 cpu_manager.go:414] "containerMap: removing container" podUID="017eaed3-ec30-4831-b3eb-86ac16a340af" containerName="payload-processing"
Oct 30 19:40:40.117114 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.117113    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="017eaed3-ec30-4831-b3eb-86ac16a340af" containerName="payload-processing"
Oct 30 19:40:40.117743 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.117429    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="017eaed3-ec30-4831-b3eb-86ac16a340af" containerName="payload-processing"
Oct 30 19:40:40.120471 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.120447    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/017eaed3-ec30-4831-b3eb-86ac16a340af-kube-api-access-7dn92\" (UniqueName: \"kubernetes.io/projected/017eaed3-ec30-4831-b3eb-86ac16a340af-kube-api-access-7dn92\") pod \"017eaed3-ec30-4831-b3eb-86ac16a340af\" (UID: \"017eaed3-ec30-4831-b3eb-86ac16a340af\") "
Oct 30 19:40:40.120611 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.120520    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/017eaed3-ec30-4831-b3eb-86ac16a340af-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/017eaed3-ec30-4831-b3eb-86ac16a340af-plugins-config-volume\") pod \"017eaed3-ec30-4831-b3eb-86ac16a340af\" (UID: \"017eaed3-ec30-4831-b3eb-86ac16a340af\") "
Oct 30 19:40:40.120993 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.120964    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/017eaed3-ec30-4831-b3eb-86ac16a340af-plugins-config-volume" pod "017eaed3-ec30-4831-b3eb-86ac16a340af" (UID: "017eaed3-ec30-4831-b3eb-86ac16a340af"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:40:40.121611 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.121586    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/017eaed3-ec30-4831-b3eb-86ac16a340af-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:40.121976 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.121954    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.122798 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.122775    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/017eaed3-ec30-4831-b3eb-86ac16a340af-kube-api-access-7dn92" pod "017eaed3-ec30-4831-b3eb-86ac16a340af" (UID: "017eaed3-ec30-4831-b3eb-86ac16a340af"). InnerVolumeSpecName "kube-api-access-7dn92". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:40:40.136748 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.136720    4524 generic.go:356] "Generic (PLEG): container finished" podID="017eaed3-ec30-4831-b3eb-86ac16a340af" containerID="0d5ef1781479ecb805b240150da8ca40eed300d79e8c17caca42af1e54a685fe" exitCode=0
Oct 30 19:40:40.136998 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.136974    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n" event={"ID":"017eaed3-ec30-4831-b3eb-86ac16a340af","Type":"ContainerDied","Data":"0d5ef1781479ecb805b240150da8ca40eed300d79e8c17caca42af1e54a685fe"}
Oct 30 19:40:40.137102 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.137029    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n" event={"ID":"017eaed3-ec30-4831-b3eb-86ac16a340af","Type":"ContainerDied","Data":"1ae87fc727e1c874f1241d37a8faa7e62da2d011a470927947b2b8db78605946"}
Oct 30 19:40:40.137102 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.137076    4524 scope.go:122] "RemoveContainer" containerID="0d5ef1781479ecb805b240150da8ca40eed300d79e8c17caca42af1e54a685fe"
Oct 30 19:40:40.137370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.137251    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n"
Oct 30 19:40:40.159255 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.158498    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"]
Oct 30 19:40:40.169787 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.167334    4524 scope.go:122] "RemoveContainer" containerID="0d5ef1781479ecb805b240150da8ca40eed300d79e8c17caca42af1e54a685fe"
Oct 30 19:40:40.175052 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:40:40.175017    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d5ef1781479ecb805b240150da8ca40eed300d79e8c17caca42af1e54a685fe\": container with ID starting with 0d5ef1781479ecb805b240150da8ca40eed300d79e8c17caca42af1e54a685fe not found: ID does not exist" containerID="0d5ef1781479ecb805b240150da8ca40eed300d79e8c17caca42af1e54a685fe"
Oct 30 19:40:40.175160 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.175062    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d5ef1781479ecb805b240150da8ca40eed300d79e8c17caca42af1e54a685fe"} err="failed to get container status \"0d5ef1781479ecb805b240150da8ca40eed300d79e8c17caca42af1e54a685fe\": rpc error: code = NotFound desc = could not find container \"0d5ef1781479ecb805b240150da8ca40eed300d79e8c17caca42af1e54a685fe\": container with ID starting with 0d5ef1781479ecb805b240150da8ca40eed300d79e8c17caca42af1e54a685fe not found: ID does not exist"
Oct 30 19:40:40.224632 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.223448    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caf02c37-523d-43ba-b171-4bcf692ed088-istio-podinfo\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.224632 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.223497    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caf02c37-523d-43ba-b171-4bcf692ed088-istio-token\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.224632 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.223582    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-istio-data\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.224632 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.223713    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-credential-socket\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.224632 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.223753    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caf02c37-523d-43ba-b171-4bcf692ed088-istiod-ca-cert\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.224632 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.224353    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svmff\" (UniqueName: \"kubernetes.io/projected/caf02c37-523d-43ba-b171-4bcf692ed088-kube-api-access-svmff\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.224632 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.224483    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-workload-socket\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.224632 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.224540    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-workload-certs\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.225054 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.224691    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-istio-envoy\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.225054 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.224974    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dn92\" (UniqueName: \"kubernetes.io/projected/017eaed3-ec30-4831-b3eb-86ac16a340af-kube-api-access-7dn92\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:40:40.240424 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.240386    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n"]
Oct 30 19:40:40.249779 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.249753    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n"]
Oct 30 19:40:40.326775 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.326705    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-istio-envoy\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.326775 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.326763    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caf02c37-523d-43ba-b171-4bcf692ed088-istio-podinfo\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.326979 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.326787    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caf02c37-523d-43ba-b171-4bcf692ed088-istio-token\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.326979 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.326811    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-istio-data\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.326979 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.326883    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-credential-socket\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.326979 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.326907    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caf02c37-523d-43ba-b171-4bcf692ed088-istiod-ca-cert\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.327223 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.327010    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svmff\" (UniqueName: \"kubernetes.io/projected/caf02c37-523d-43ba-b171-4bcf692ed088-kube-api-access-svmff\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.327223 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.327055    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-workload-socket\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.327223 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.327111    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-workload-certs\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.327369 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.327298    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-istio-data\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.327369 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.327348    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-credential-socket\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.327559 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.327534    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-workload-socket\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.327616 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.327555    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-workload-certs\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.327769 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.327751    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caf02c37-523d-43ba-b171-4bcf692ed088-istiod-ca-cert\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.329108 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.329088    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-istio-envoy\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.329397 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.329378    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caf02c37-523d-43ba-b171-4bcf692ed088-istio-podinfo\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.338633 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.338608    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caf02c37-523d-43ba-b171-4bcf692ed088-istio-token\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.338713 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.338608    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svmff\" (UniqueName: \"kubernetes.io/projected/caf02c37-523d-43ba-b171-4bcf692ed088-kube-api-access-svmff\") pod \"e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") " pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.396571 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.396538    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="017eaed3-ec30-4831-b3eb-86ac16a340af" path="/var/lib/kubelet/pods/017eaed3-ec30-4831-b3eb-86ac16a340af/volumes"
Oct 30 19:40:40.477203 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.477142    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:40.783819 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:40:40.783773    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaf02c37_523d_43ba_b171_4bcf692ed088.slice/crio-2418af1dde3b897fb45f03bff3cae3139e3f9e07cf079aa5e58db2e664e50058 WatchSource:0}: Error finding container 2418af1dde3b897fb45f03bff3cae3139e3f9e07cf079aa5e58db2e664e50058: Status 404 returned error can't find the container with id 2418af1dde3b897fb45f03bff3cae3139e3f9e07cf079aa5e58db2e664e50058
Oct 30 19:40:40.785399 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.785374    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"]
Oct 30 19:40:40.787660 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.787631    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:40:40.787744 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.787708    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:40:40.787804 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.787745    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:40:40.860397 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:40.860328    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-ipp-a-7e6199-6ff9868fc4-dqp4n" podUID="017eaed3-ec30-4831-b3eb-86ac16a340af" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.164:50052\" within 1s: context deadline exceeded"
Oct 30 19:40:41.156908 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:41.156871    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5" event={"ID":"caf02c37-523d-43ba-b171-4bcf692ed088","Type":"ContainerStarted","Data":"0f8e67d6ad3da72cca4fce37ffa54b98fa2f9130df826dc61b5ec294f0f732a6"}
Oct 30 19:40:41.157148 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:41.156927    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5" event={"ID":"caf02c37-523d-43ba-b171-4bcf692ed088","Type":"ContainerStarted","Data":"2418af1dde3b897fb45f03bff3cae3139e3f9e07cf079aa5e58db2e664e50058"}
Oct 30 19:40:41.191450 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:41.191383    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5" podStartSLOduration=116.191363416 podStartE2EDuration="1m56.191363416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:41.180006738 +0000 UTC m=+2072.920997841" watchObservedRunningTime="2026-10-30 19:40:41.191363416 +0000 UTC m=+2072.932354494"
Oct 30 19:40:41.477851 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:41.477760    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:41.479331 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:41.479303    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 19:40:41.479473 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:41.479359    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5" podUID="caf02c37-523d-43ba-b171-4bcf692ed088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 19:40:42.487192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:42.486623    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 19:40:42.487192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:42.486710    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5" podUID="caf02c37-523d-43ba-b171-4bcf692ed088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 19:40:43.483282 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:43.483234    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 19:40:43.483576 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:43.483534    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5" podUID="caf02c37-523d-43ba-b171-4bcf692ed088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 19:40:44.483194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:44.482777    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 19:40:44.483194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:44.482832    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5" podUID="caf02c37-523d-43ba-b171-4bcf692ed088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 19:40:45.480195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:45.479556    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 19:40:45.480195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:45.479620    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5" podUID="caf02c37-523d-43ba-b171-4bcf692ed088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 19:40:45.642423 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:40:45.642344    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:40:46.477552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:46.477516    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 19:40:46.477857 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:46.477829    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5" podUID="caf02c37-523d-43ba-b171-4bcf692ed088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 19:40:47.478341 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:47.478305    4524 patch_prober.go:29] interesting pod/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused" start-of-body=
Oct 30 19:40:47.478719 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:47.478373    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5" podUID="caf02c37-523d-43ba-b171-4bcf692ed088" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.167:15021/healthz/ready\": dial tcp 10.133.0.167:15021: connect: connection refused"
Oct 30 19:40:48.483714 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:48.483678    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:48.484203 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:48.484110    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:40:48.484917 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:40:48.484895    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:41:15.772231 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:41:15.771655    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:41:22.218848 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:22.218770    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"]
Oct 30 19:41:22.219363 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:22.218987    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc" podUID="d72b215c-1d3b-400c-9238-906b0199f4c7" containerName="istio-proxy" containerID="cri-o://47ce3a2c33bc18d4f462cdf8af896c1076f1b432f8f2d4e42f5d27d545f8a72d" gracePeriod=30
Oct 30 19:41:27.529302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.529271    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:41:27.573037 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.573003    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"]
Oct 30 19:41:27.574206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.574150    4524 cpu_manager.go:414] "containerMap: removing container" podUID="d72b215c-1d3b-400c-9238-906b0199f4c7" containerName="istio-proxy"
Oct 30 19:41:27.574206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.574190    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d72b215c-1d3b-400c-9238-906b0199f4c7" containerName="istio-proxy"
Oct 30 19:41:27.574337 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.574330    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="d72b215c-1d3b-400c-9238-906b0199f4c7" containerName="istio-proxy"
Oct 30 19:41:27.578449 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.578425    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.590435 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.590396    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-token\" (UniqueName: \"kubernetes.io/projected/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-token\") pod \"d72b215c-1d3b-400c-9238-906b0199f4c7\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") "
Oct 30 19:41:27.591017 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.590673    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d72b215c-1d3b-400c-9238-906b0199f4c7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d72b215c-1d3b-400c-9238-906b0199f4c7-istiod-ca-cert\") pod \"d72b215c-1d3b-400c-9238-906b0199f4c7\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") "
Oct 30 19:41:27.591705 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.591684    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-credential-socket\") pod \"d72b215c-1d3b-400c-9238-906b0199f4c7\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") "
Oct 30 19:41:27.591935 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.591901    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d72b215c-1d3b-400c-9238-906b0199f4c7-kube-api-access-bb42l\" (UniqueName: \"kubernetes.io/projected/d72b215c-1d3b-400c-9238-906b0199f4c7-kube-api-access-bb42l\") pod \"d72b215c-1d3b-400c-9238-906b0199f4c7\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") "
Oct 30 19:41:27.592142 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.592118    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-data\") pod \"d72b215c-1d3b-400c-9238-906b0199f4c7\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") "
Oct 30 19:41:27.592310 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.592203    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-workload-certs\") pod \"d72b215c-1d3b-400c-9238-906b0199f4c7\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") "
Oct 30 19:41:27.592310 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.592276    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-podinfo\") pod \"d72b215c-1d3b-400c-9238-906b0199f4c7\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") "
Oct 30 19:41:27.592310 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.592303    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-workload-socket\") pod \"d72b215c-1d3b-400c-9238-906b0199f4c7\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") "
Oct 30 19:41:27.592487 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.592442    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-envoy\") pod \"d72b215c-1d3b-400c-9238-906b0199f4c7\" (UID: \"d72b215c-1d3b-400c-9238-906b0199f4c7\") "
Oct 30 19:41:27.609614 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.601618    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-e50648-openshift-default-dockercfg-k79r8\""
Oct 30 19:41:27.609614 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.605707    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-data" pod "d72b215c-1d3b-400c-9238-906b0199f4c7" (UID: "d72b215c-1d3b-400c-9238-906b0199f4c7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:27.609614 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.605974    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-workload-socket" pod "d72b215c-1d3b-400c-9238-906b0199f4c7" (UID: "d72b215c-1d3b-400c-9238-906b0199f4c7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:27.609614 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.606098    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-workload-certs" pod "d72b215c-1d3b-400c-9238-906b0199f4c7" (UID: "d72b215c-1d3b-400c-9238-906b0199f4c7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:27.609614 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.606125    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-credential-socket" pod "d72b215c-1d3b-400c-9238-906b0199f4c7" (UID: "d72b215c-1d3b-400c-9238-906b0199f4c7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:27.609614 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.606352    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d72b215c-1d3b-400c-9238-906b0199f4c7-istiod-ca-cert" pod "d72b215c-1d3b-400c-9238-906b0199f4c7" (UID: "d72b215c-1d3b-400c-9238-906b0199f4c7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:41:27.612135 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.612109    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"]
Oct 30 19:41:27.617656 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.617632    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-envoy" pod "d72b215c-1d3b-400c-9238-906b0199f4c7" (UID: "d72b215c-1d3b-400c-9238-906b0199f4c7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:41:27.619621 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.618757    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d72b215c-1d3b-400c-9238-906b0199f4c7-kube-api-access-bb42l" pod "d72b215c-1d3b-400c-9238-906b0199f4c7" (UID: "d72b215c-1d3b-400c-9238-906b0199f4c7"). InnerVolumeSpecName "kube-api-access-bb42l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:27.620659 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.620625    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-podinfo" pod "d72b215c-1d3b-400c-9238-906b0199f4c7" (UID: "d72b215c-1d3b-400c-9238-906b0199f4c7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:41:27.621666 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.621640    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-token" pod "d72b215c-1d3b-400c-9238-906b0199f4c7" (UID: "d72b215c-1d3b-400c-9238-906b0199f4c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:41:27.720199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.696334    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bxgr\" (UniqueName: \"kubernetes.io/projected/98c13946-1bdd-4129-9b78-88c56659f432-kube-api-access-8bxgr\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.721468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.721397    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-credential-socket\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.721617 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.721500    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-istio-envoy\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.721617 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.721553    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-workload-socket\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.721741 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.721632    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-workload-certs\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.721741 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.721688    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-istio-data\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.722233 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.721844    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98c13946-1bdd-4129-9b78-88c56659f432-istio-token\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.722233 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.721922    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98c13946-1bdd-4129-9b78-88c56659f432-istio-podinfo\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.722233 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.722028    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98c13946-1bdd-4129-9b78-88c56659f432-istiod-ca-cert\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.722454 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.722287    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:41:27.722454 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.722321    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d72b215c-1d3b-400c-9238-906b0199f4c7-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:41:27.722454 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.722337    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:41:27.722454 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.722351    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bb42l\" (UniqueName: \"kubernetes.io/projected/d72b215c-1d3b-400c-9238-906b0199f4c7-kube-api-access-bb42l\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:41:27.722454 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.722368    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:41:27.722454 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.722383    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:41:27.722454 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.722397    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:41:27.722454 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.722412    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:41:27.722454 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.722428    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d72b215c-1d3b-400c-9238-906b0199f4c7-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:41:27.823556 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.823452    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-workload-certs\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.823954 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.823917    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-istio-data\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.824124 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.824098    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-workload-certs\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.824442 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.824419    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98c13946-1bdd-4129-9b78-88c56659f432-istio-token\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.824670 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.824653    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98c13946-1bdd-4129-9b78-88c56659f432-istio-podinfo\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.824849 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.824489    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-istio-data\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.825087 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.825069    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98c13946-1bdd-4129-9b78-88c56659f432-istiod-ca-cert\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.825325 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.825299    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bxgr\" (UniqueName: \"kubernetes.io/projected/98c13946-1bdd-4129-9b78-88c56659f432-kube-api-access-8bxgr\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.825474 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.825450    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-credential-socket\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.825646 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.825619    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-istio-envoy\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.825769 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.825755    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-workload-socket\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.825926 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.825901    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98c13946-1bdd-4129-9b78-88c56659f432-istiod-ca-cert\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.826217 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.826194    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-credential-socket\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.828734 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.828701    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-workload-socket\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.831257 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.831156    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98c13946-1bdd-4129-9b78-88c56659f432-istio-podinfo\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.845188 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.843393    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-istio-envoy\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.854490 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.854423    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bxgr\" (UniqueName: \"kubernetes.io/projected/98c13946-1bdd-4129-9b78-88c56659f432-kube-api-access-8bxgr\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.857252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.857150    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98c13946-1bdd-4129-9b78-88c56659f432-istio-token\") pod \"e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") " pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:27.938337 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:27.938287    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:28.196983 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:28.196946    4524 generic.go:356] "Generic (PLEG): container finished" podID="d72b215c-1d3b-400c-9238-906b0199f4c7" containerID="47ce3a2c33bc18d4f462cdf8af896c1076f1b432f8f2d4e42f5d27d545f8a72d" exitCode=0
Oct 30 19:41:28.197165 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:28.196999    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"
Oct 30 19:41:28.197165 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:28.197039    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc" event={"ID":"d72b215c-1d3b-400c-9238-906b0199f4c7","Type":"ContainerDied","Data":"47ce3a2c33bc18d4f462cdf8af896c1076f1b432f8f2d4e42f5d27d545f8a72d"}
Oct 30 19:41:28.197165 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:28.197118    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc" event={"ID":"d72b215c-1d3b-400c-9238-906b0199f4c7","Type":"ContainerDied","Data":"67617be13bda60b93f3c01a7cf054aeb43854f4efc85f14253b934047394e501"}
Oct 30 19:41:28.197165 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:28.197147    4524 scope.go:122] "RemoveContainer" containerID="47ce3a2c33bc18d4f462cdf8af896c1076f1b432f8f2d4e42f5d27d545f8a72d"
Oct 30 19:41:28.210960 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:28.210940    4524 scope.go:122] "RemoveContainer" containerID="47ce3a2c33bc18d4f462cdf8af896c1076f1b432f8f2d4e42f5d27d545f8a72d"
Oct 30 19:41:28.211245 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:41:28.211218    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47ce3a2c33bc18d4f462cdf8af896c1076f1b432f8f2d4e42f5d27d545f8a72d\": container with ID starting with 47ce3a2c33bc18d4f462cdf8af896c1076f1b432f8f2d4e42f5d27d545f8a72d not found: ID does not exist" containerID="47ce3a2c33bc18d4f462cdf8af896c1076f1b432f8f2d4e42f5d27d545f8a72d"
Oct 30 19:41:28.211343 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:28.211248    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47ce3a2c33bc18d4f462cdf8af896c1076f1b432f8f2d4e42f5d27d545f8a72d"} err="failed to get container status \"47ce3a2c33bc18d4f462cdf8af896c1076f1b432f8f2d4e42f5d27d545f8a72d\": rpc error: code = NotFound desc = could not find container \"47ce3a2c33bc18d4f462cdf8af896c1076f1b432f8f2d4e42f5d27d545f8a72d\": container with ID starting with 47ce3a2c33bc18d4f462cdf8af896c1076f1b432f8f2d4e42f5d27d545f8a72d not found: ID does not exist"
Oct 30 19:41:28.230723 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:28.230656    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"]
Oct 30 19:41:28.236058 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:28.236035    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-4rhhc"]
Oct 30 19:41:28.242124 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:28.242104    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"]
Oct 30 19:41:28.244067 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:41:28.244041    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98c13946_1bdd_4129_9b78_88c56659f432.slice/crio-84e61d8d3efc7028dbfa22b50a7ffd08676fc800734e00ec9d321a40afdd62fb WatchSource:0}: Error finding container 84e61d8d3efc7028dbfa22b50a7ffd08676fc800734e00ec9d321a40afdd62fb: Status 404 returned error can't find the container with id 84e61d8d3efc7028dbfa22b50a7ffd08676fc800734e00ec9d321a40afdd62fb
Oct 30 19:41:28.247306 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:28.247269    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:41:28.247409 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:28.247352    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:41:28.247409 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:28.247394    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:41:28.447014 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:28.446959    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d72b215c-1d3b-400c-9238-906b0199f4c7" path="/var/lib/kubelet/pods/d72b215c-1d3b-400c-9238-906b0199f4c7/volumes"
Oct 30 19:41:29.208245 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:29.208208    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg" event={"ID":"98c13946-1bdd-4129-9b78-88c56659f432","Type":"ContainerStarted","Data":"69cbdc3aeed23a0bd335cc14eb2a3e7e5b7894532dd5b047b2ab9eda6f9aae49"}
Oct 30 19:41:29.208750 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:29.208274    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg" event={"ID":"98c13946-1bdd-4129-9b78-88c56659f432","Type":"ContainerStarted","Data":"84e61d8d3efc7028dbfa22b50a7ffd08676fc800734e00ec9d321a40afdd62fb"}
Oct 30 19:41:29.259195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:29.255216    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg" podStartSLOduration=13.255094657 podStartE2EDuration="13.255094657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:41:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:41:29.235916221 +0000 UTC m=+2120.976907302" watchObservedRunningTime="2026-10-30 19:41:29.255094657 +0000 UTC m=+2120.996085762"
Oct 30 19:41:29.938612 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:29.938575    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:29.945104 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:29.945055    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:30.259243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:30.259114    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:30.270704 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:41:30.261626    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:41:45.648339 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:41:45.648299    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:42:15.628846 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:42:15.628808    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:42:35.760784 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:35.760751    4524 generic.go:356] "Generic (PLEG): container finished" podID="98c13946-1bdd-4129-9b78-88c56659f432" containerID="69cbdc3aeed23a0bd335cc14eb2a3e7e5b7894532dd5b047b2ab9eda6f9aae49" exitCode=0
Oct 30 19:42:35.761356 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:35.761048    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg" event={"ID":"98c13946-1bdd-4129-9b78-88c56659f432","Type":"ContainerDied","Data":"69cbdc3aeed23a0bd335cc14eb2a3e7e5b7894532dd5b047b2ab9eda6f9aae49"}
Oct 30 19:42:35.761625 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:35.761594    4524 scope.go:122] "RemoveContainer" containerID="69cbdc3aeed23a0bd335cc14eb2a3e7e5b7894532dd5b047b2ab9eda6f9aae49"
Oct 30 19:42:35.764855 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:35.764459    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:42:35.764855 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:35.764557    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:42:35.764855 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:35.764610    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:42:36.774234 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:36.774191    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg" event={"ID":"98c13946-1bdd-4129-9b78-88c56659f432","Type":"ContainerStarted","Data":"3b12f1a25d91ac5434e5f60ff5620e1cd65ad22947861557ababa80e1817568c"}
Oct 30 19:42:36.938784 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:36.938745    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:42:36.938964 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:36.938878    4524 patch_prober.go:29] interesting pod/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.168:15021/healthz/ready\": dial tcp 10.133.0.168:15021: connect: connection refused" start-of-body=
Oct 30 19:42:36.938964 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:36.938938    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg" podUID="98c13946-1bdd-4129-9b78-88c56659f432" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.168:15021/healthz/ready\": dial tcp 10.133.0.168:15021: connect: connection refused"
Oct 30 19:42:37.938877 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:37.938845    4524 patch_prober.go:29] interesting pod/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.168:15021/healthz/ready\": dial tcp 10.133.0.168:15021: connect: connection refused" start-of-body=
Oct 30 19:42:37.939283 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:37.938892    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg" podUID="98c13946-1bdd-4129-9b78-88c56659f432" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.168:15021/healthz/ready\": dial tcp 10.133.0.168:15021: connect: connection refused"
Oct 30 19:42:38.938639 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:38.938607    4524 patch_prober.go:29] interesting pod/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.168:15021/healthz/ready\": dial tcp 10.133.0.168:15021: connect: connection refused" start-of-body=
Oct 30 19:42:38.938817 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:38.938656    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg" podUID="98c13946-1bdd-4129-9b78-88c56659f432" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.168:15021/healthz/ready\": dial tcp 10.133.0.168:15021: connect: connection refused"
Oct 30 19:42:39.939093 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:39.939063    4524 patch_prober.go:29] interesting pod/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.168:15021/healthz/ready\": dial tcp 10.133.0.168:15021: connect: connection refused" start-of-body=
Oct 30 19:42:39.939479 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:39.939117    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg" podUID="98c13946-1bdd-4129-9b78-88c56659f432" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.168:15021/healthz/ready\": dial tcp 10.133.0.168:15021: connect: connection refused"
Oct 30 19:42:40.939427 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:40.939393    4524 patch_prober.go:29] interesting pod/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.168:15021/healthz/ready\": dial tcp 10.133.0.168:15021: connect: connection refused" start-of-body=
Oct 30 19:42:40.940129 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:40.940097    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg" podUID="98c13946-1bdd-4129-9b78-88c56659f432" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.168:15021/healthz/ready\": dial tcp 10.133.0.168:15021: connect: connection refused"
Oct 30 19:42:41.939295 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:41.939253    4524 patch_prober.go:29] interesting pod/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.168:15021/healthz/ready\": dial tcp 10.133.0.168:15021: connect: connection refused" start-of-body=
Oct 30 19:42:41.939487 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:41.939351    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg" podUID="98c13946-1bdd-4129-9b78-88c56659f432" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.168:15021/healthz/ready\": dial tcp 10.133.0.168:15021: connect: connection refused"
Oct 30 19:42:42.944001 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:42.943968    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:42:42.944534 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:42.944396    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:42:42.945218 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:42:42.945164    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:42:45.749834 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:42:45.749793    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:43:13.722564 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:13.722531    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"]
Oct 30 19:43:13.723285 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:13.722772    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs" podUID="95bd6255-4eb5-40ba-a80b-9e387b4c5c3a" containerName="maas-api" containerID="cri-o://af54ef1f63be9305f9f565ee3a084a6dd652fc3440e33d72eb5a38f6469e2ed2" gracePeriod=30
Oct 30 19:43:14.293097 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.293056    4524 generic.go:356] "Generic (PLEG): container finished" podID="95bd6255-4eb5-40ba-a80b-9e387b4c5c3a" containerID="af54ef1f63be9305f9f565ee3a084a6dd652fc3440e33d72eb5a38f6469e2ed2" exitCode=0
Oct 30 19:43:14.293375 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.293116    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs" event={"ID":"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a","Type":"ContainerDied","Data":"af54ef1f63be9305f9f565ee3a084a6dd652fc3440e33d72eb5a38f6469e2ed2"}
Oct 30 19:43:14.520294 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.517548    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"
Oct 30 19:43:14.583085 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.583053    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"]
Oct 30 19:43:14.584234 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.584213    4524 cpu_manager.go:414] "containerMap: removing container" podUID="95bd6255-4eb5-40ba-a80b-9e387b4c5c3a" containerName="maas-api"
Oct 30 19:43:14.584401 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.584390    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95bd6255-4eb5-40ba-a80b-9e387b4c5c3a" containerName="maas-api"
Oct 30 19:43:14.584737 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.584725    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="95bd6255-4eb5-40ba-a80b-9e387b4c5c3a" containerName="maas-api"
Oct 30 19:43:14.587783 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.587760    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"
Oct 30 19:43:14.598069 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.595540    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-e50648-serving-cert\""
Oct 30 19:43:14.608718 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.607017    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"]
Oct 30 19:43:14.695052 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.694803    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-kube-api-access-bmjjf\" (UniqueName: \"kubernetes.io/projected/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-kube-api-access-bmjjf\") pod \"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a\" (UID: \"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a\") "
Oct 30 19:43:14.695331 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.695102    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-metrics-certs\") pod \"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a\" (UID: \"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a\") "
Oct 30 19:43:14.695331 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.695215    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-maas-api-tls\") pod \"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a\" (UID: \"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a\") "
Oct 30 19:43:14.695331 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.695283    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-openshift-service-ca\") pod \"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a\" (UID: \"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a\") "
Oct 30 19:43:14.695708 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.695673    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a26c2511-7fe6-4a60-86f8-781c17fac337-metrics-certs\") pod \"maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc\" (UID: \"a26c2511-7fe6-4a60-86f8-781c17fac337\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"
Oct 30 19:43:14.695859 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.695719    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a26c2511-7fe6-4a60-86f8-781c17fac337-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc\" (UID: \"a26c2511-7fe6-4a60-86f8-781c17fac337\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"
Oct 30 19:43:14.695859 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.695846    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a26c2511-7fe6-4a60-86f8-781c17fac337-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc\" (UID: \"a26c2511-7fe6-4a60-86f8-781c17fac337\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"
Oct 30 19:43:14.696018 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.695998    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h277g\" (UniqueName: \"kubernetes.io/projected/a26c2511-7fe6-4a60-86f8-781c17fac337-kube-api-access-h277g\") pod \"maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc\" (UID: \"a26c2511-7fe6-4a60-86f8-781c17fac337\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"
Oct 30 19:43:14.698341 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.697159    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-openshift-service-ca" pod "95bd6255-4eb5-40ba-a80b-9e387b4c5c3a" (UID: "95bd6255-4eb5-40ba-a80b-9e387b4c5c3a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:14.699054 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.699027    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-metrics-certs" pod "95bd6255-4eb5-40ba-a80b-9e387b4c5c3a" (UID: "95bd6255-4eb5-40ba-a80b-9e387b4c5c3a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:43:14.699142 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.699083    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-maas-api-tls" pod "95bd6255-4eb5-40ba-a80b-9e387b4c5c3a" (UID: "95bd6255-4eb5-40ba-a80b-9e387b4c5c3a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:43:14.699233 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.699149    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-kube-api-access-bmjjf" pod "95bd6255-4eb5-40ba-a80b-9e387b4c5c3a" (UID: "95bd6255-4eb5-40ba-a80b-9e387b4c5c3a"). InnerVolumeSpecName "kube-api-access-bmjjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:14.797892 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.797853    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a26c2511-7fe6-4a60-86f8-781c17fac337-metrics-certs\") pod \"maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc\" (UID: \"a26c2511-7fe6-4a60-86f8-781c17fac337\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"
Oct 30 19:43:14.798372 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.797905    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a26c2511-7fe6-4a60-86f8-781c17fac337-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc\" (UID: \"a26c2511-7fe6-4a60-86f8-781c17fac337\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"
Oct 30 19:43:14.798372 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.797962    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a26c2511-7fe6-4a60-86f8-781c17fac337-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc\" (UID: \"a26c2511-7fe6-4a60-86f8-781c17fac337\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"
Oct 30 19:43:14.798372 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.798058    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h277g\" (UniqueName: \"kubernetes.io/projected/a26c2511-7fe6-4a60-86f8-781c17fac337-kube-api-access-h277g\") pod \"maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc\" (UID: \"a26c2511-7fe6-4a60-86f8-781c17fac337\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"
Oct 30 19:43:14.798372 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.798148    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:14.798372 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.798164    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:14.798372 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.798194    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmjjf\" (UniqueName: \"kubernetes.io/projected/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-kube-api-access-bmjjf\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:14.798372 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.798211    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:14.798741 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.798719    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a26c2511-7fe6-4a60-86f8-781c17fac337-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc\" (UID: \"a26c2511-7fe6-4a60-86f8-781c17fac337\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"
Oct 30 19:43:14.800673 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.800642    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a26c2511-7fe6-4a60-86f8-781c17fac337-metrics-certs\") pod \"maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc\" (UID: \"a26c2511-7fe6-4a60-86f8-781c17fac337\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"
Oct 30 19:43:14.800673 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.800667    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a26c2511-7fe6-4a60-86f8-781c17fac337-maas-api-tls\") pod \"maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc\" (UID: \"a26c2511-7fe6-4a60-86f8-781c17fac337\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"
Oct 30 19:43:14.810029 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.810000    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h277g\" (UniqueName: \"kubernetes.io/projected/a26c2511-7fe6-4a60-86f8-781c17fac337-kube-api-access-h277g\") pod \"maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc\" (UID: \"a26c2511-7fe6-4a60-86f8-781c17fac337\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"
Oct 30 19:43:14.911910 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:14.911874    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"
Oct 30 19:43:15.204017 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:43:15.203976    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda26c2511_7fe6_4a60_86f8_781c17fac337.slice/crio-388b3a5d4fd343a47af1be40000c054b7b6d9ad09f5af892e344274fda4e35ad WatchSource:0}: Error finding container 388b3a5d4fd343a47af1be40000c054b7b6d9ad09f5af892e344274fda4e35ad: Status 404 returned error can't find the container with id 388b3a5d4fd343a47af1be40000c054b7b6d9ad09f5af892e344274fda4e35ad
Oct 30 19:43:15.205863 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:15.205843    4524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:43:15.206070 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:15.206046    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"]
Oct 30 19:43:15.304246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:15.304120    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs" event={"ID":"95bd6255-4eb5-40ba-a80b-9e387b4c5c3a","Type":"ContainerDied","Data":"b3ea516bc0cd5e80b62d5f50cde362c9d1b2e3b44352c93cde3b7b35155872cf"}
Oct 30 19:43:15.304246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:15.304133    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"
Oct 30 19:43:15.304246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:15.304199    4524 scope.go:122] "RemoveContainer" containerID="af54ef1f63be9305f9f565ee3a084a6dd652fc3440e33d72eb5a38f6469e2ed2"
Oct 30 19:43:15.305699 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:15.305671    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc" event={"ID":"a26c2511-7fe6-4a60-86f8-781c17fac337","Type":"ContainerStarted","Data":"388b3a5d4fd343a47af1be40000c054b7b6d9ad09f5af892e344274fda4e35ad"}
Oct 30 19:43:15.336709 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:15.336689    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"]
Oct 30 19:43:15.341190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:15.341147    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-69c573-5b8c4b58dd-mpfcs"]
Oct 30 19:43:15.640204 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:43:15.632450    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:43:16.354601 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:16.354111    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc" event={"ID":"a26c2511-7fe6-4a60-86f8-781c17fac337","Type":"ContainerStarted","Data":"ca99ebc94faea5f62cdb83ed0be289349a2db2368e4268c297f0d712c6ae3452"}
Oct 30 19:43:16.354601 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:16.354332    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"
Oct 30 19:43:16.380259 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:16.380202    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc" podStartSLOduration=84.889554164 podStartE2EDuration="1m25.380182832s" totalImagesPullingTime="490.628668ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:41:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:16.375085567 +0000 UTC m=+2228.116076670" watchObservedRunningTime="2026-10-30 19:43:16.380182832 +0000 UTC m=+2228.121173903"
Oct 30 19:43:16.392981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:16.392947    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="95bd6255-4eb5-40ba-a80b-9e387b4c5c3a" path="/var/lib/kubelet/pods/95bd6255-4eb5-40ba-a80b-9e387b4c5c3a/volumes"
Oct 30 19:43:22.368775 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:22.368742    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"
Oct 30 19:43:40.357731 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:40.357697    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"]
Oct 30 19:43:40.358406 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:40.357930    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt" podUID="f88b8b61-5083-47c0-821d-92bf69941379" containerName="maas-api" containerID="cri-o://14e57a57596a92cd3ac3e229a331d29e446591eed22b613a2a9b3db7957e079a" gracePeriod=30
Oct 30 19:43:40.952432 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:40.951607    4524 generic.go:356] "Generic (PLEG): container finished" podID="f88b8b61-5083-47c0-821d-92bf69941379" containerID="14e57a57596a92cd3ac3e229a331d29e446591eed22b613a2a9b3db7957e079a" exitCode=0
Oct 30 19:43:40.952432 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:40.951698    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt" event={"ID":"f88b8b61-5083-47c0-821d-92bf69941379","Type":"ContainerDied","Data":"14e57a57596a92cd3ac3e229a331d29e446591eed22b613a2a9b3db7957e079a"}
Oct 30 19:43:41.035676 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.034381    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"
Oct 30 19:43:41.036144 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.036111    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f88b8b61-5083-47c0-821d-92bf69941379-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f88b8b61-5083-47c0-821d-92bf69941379-maas-api-tls\") pod \"f88b8b61-5083-47c0-821d-92bf69941379\" (UID: \"f88b8b61-5083-47c0-821d-92bf69941379\") "
Oct 30 19:43:41.036343 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.036300    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f88b8b61-5083-47c0-821d-92bf69941379-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f88b8b61-5083-47c0-821d-92bf69941379-openshift-service-ca\") pod \"f88b8b61-5083-47c0-821d-92bf69941379\" (UID: \"f88b8b61-5083-47c0-821d-92bf69941379\") "
Oct 30 19:43:41.036416 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.036360    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f88b8b61-5083-47c0-821d-92bf69941379-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f88b8b61-5083-47c0-821d-92bf69941379-metrics-certs\") pod \"f88b8b61-5083-47c0-821d-92bf69941379\" (UID: \"f88b8b61-5083-47c0-821d-92bf69941379\") "
Oct 30 19:43:41.037230 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.037203    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f88b8b61-5083-47c0-821d-92bf69941379-openshift-service-ca" pod "f88b8b61-5083-47c0-821d-92bf69941379" (UID: "f88b8b61-5083-47c0-821d-92bf69941379"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:41.042276 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.042241    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f88b8b61-5083-47c0-821d-92bf69941379-metrics-certs" pod "f88b8b61-5083-47c0-821d-92bf69941379" (UID: "f88b8b61-5083-47c0-821d-92bf69941379"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:43:41.042276 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.042254    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f88b8b61-5083-47c0-821d-92bf69941379-maas-api-tls" pod "f88b8b61-5083-47c0-821d-92bf69941379" (UID: "f88b8b61-5083-47c0-821d-92bf69941379"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:43:41.043731 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.043701    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f88b8b61-5083-47c0-821d-92bf69941379-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:41.043909 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.043893    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f88b8b61-5083-47c0-821d-92bf69941379-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:41.044039 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.044024    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f88b8b61-5083-47c0-821d-92bf69941379-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:41.074274 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.074244    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"]
Oct 30 19:43:41.075600 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.075575    4524 cpu_manager.go:414] "containerMap: removing container" podUID="f88b8b61-5083-47c0-821d-92bf69941379" containerName="maas-api"
Oct 30 19:43:41.075600 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.075599    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f88b8b61-5083-47c0-821d-92bf69941379" containerName="maas-api"
Oct 30 19:43:41.075836 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.075821    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="f88b8b61-5083-47c0-821d-92bf69941379" containerName="maas-api"
Oct 30 19:43:41.080696 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.080675    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.093633 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.088347    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-7aa217-openshift-default-dockercfg-r7c4w\""
Oct 30 19:43:41.104872 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.104847    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"]
Oct 30 19:43:41.145372 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.145344    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f88b8b61-5083-47c0-821d-92bf69941379-kube-api-access-g9q9x\" (UniqueName: \"kubernetes.io/projected/f88b8b61-5083-47c0-821d-92bf69941379-kube-api-access-g9q9x\") pod \"f88b8b61-5083-47c0-821d-92bf69941379\" (UID: \"f88b8b61-5083-47c0-821d-92bf69941379\") "
Oct 30 19:43:41.145751 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.145728    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-credential-socket\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.145861 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.145846    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4jxt\" (UniqueName: \"kubernetes.io/projected/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-kube-api-access-g4jxt\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.145907 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.145898    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-workload-certs\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.146036 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.146014    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-envoy\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.146082 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.146069    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-data\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.146131 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.146119    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-workload-socket\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.146238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.146223    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-token\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.146287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.146267    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istiod-ca-cert\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.146353 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.146338    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-podinfo\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.147594 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.147573    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f88b8b61-5083-47c0-821d-92bf69941379-kube-api-access-g9q9x" pod "f88b8b61-5083-47c0-821d-92bf69941379" (UID: "f88b8b61-5083-47c0-821d-92bf69941379"). InnerVolumeSpecName "kube-api-access-g9q9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:41.254243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.252151    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-podinfo\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.254243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.252329    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-credential-socket\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.254243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.252412    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4jxt\" (UniqueName: \"kubernetes.io/projected/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-kube-api-access-g4jxt\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.254243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.252458    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-workload-certs\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.254243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.252505    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-envoy\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.254243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.252538    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-data\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.254243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.252599    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-workload-socket\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.254243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.252637    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-token\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.254243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.252670    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istiod-ca-cert\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.254243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.252748    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9q9x\" (UniqueName: \"kubernetes.io/projected/f88b8b61-5083-47c0-821d-92bf69941379-kube-api-access-g9q9x\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:41.254243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.253400    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istiod-ca-cert\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.254243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.253512    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-workload-certs\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.254243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.254095    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-credential-socket\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.254243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.254112    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-workload-socket\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.255093 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.254531    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-data\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.307190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.258100    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-envoy\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.307190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.258771    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-podinfo\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.307190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.292849    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4jxt\" (UniqueName: \"kubernetes.io/projected/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-kube-api-access-g4jxt\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.307190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.295329    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-token\") pod \"e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") " pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.402252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.402159    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:41.718121 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.718092    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"]
Oct 30 19:43:41.719519 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:43:41.719494    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod881f121f_ca0c_4a99_a9d2_d0cca6b54a48.slice/crio-f157d8fe0190f7f3d7d73c4f5c6fad062f30a9b12defd6d2448016927953a8a4 WatchSource:0}: Error finding container f157d8fe0190f7f3d7d73c4f5c6fad062f30a9b12defd6d2448016927953a8a4: Status 404 returned error can't find the container with id f157d8fe0190f7f3d7d73c4f5c6fad062f30a9b12defd6d2448016927953a8a4
Oct 30 19:43:41.722901 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.722868    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:43:41.722980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.722933    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:43:41.722980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.722965    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:43:41.962289 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.962257    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"
Oct 30 19:43:41.962451 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.962293    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt" event={"ID":"f88b8b61-5083-47c0-821d-92bf69941379","Type":"ContainerDied","Data":"80337aab057994f9d2b9e7b139c42b4b082cdfeb8fda49e08f9561039d2db24e"}
Oct 30 19:43:41.962451 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.962353    4524 scope.go:122] "RemoveContainer" containerID="14e57a57596a92cd3ac3e229a331d29e446591eed22b613a2a9b3db7957e079a"
Oct 30 19:43:41.964652 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.964622    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql" event={"ID":"881f121f-ca0c-4a99-a9d2-d0cca6b54a48","Type":"ContainerStarted","Data":"905aaceacb19f21070b83bc9cfd84e72fc956abb656dc5f788f154c479933cd5"}
Oct 30 19:43:41.964798 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.964677    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql" event={"ID":"881f121f-ca0c-4a99-a9d2-d0cca6b54a48","Type":"ContainerStarted","Data":"f157d8fe0190f7f3d7d73c4f5c6fad062f30a9b12defd6d2448016927953a8a4"}
Oct 30 19:43:42.000090 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:41.999592    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql" podStartSLOduration=5.999579475 podStartE2EDuration="5.999579475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:41.998648179 +0000 UTC m=+2253.739639256" watchObservedRunningTime="2026-10-30 19:43:41.999579475 +0000 UTC m=+2253.740570627"
Oct 30 19:43:42.018491 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:42.018462    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"]
Oct 30 19:43:42.023069 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:42.023045    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-0a6e66-65bf4f9fb4-wvjwt"]
Oct 30 19:43:42.405247 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:42.404674    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f88b8b61-5083-47c0-821d-92bf69941379" path="/var/lib/kubelet/pods/f88b8b61-5083-47c0-821d-92bf69941379/volumes"
Oct 30 19:43:42.405247 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:42.405053    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:42.408807 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:42.408784    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:43.053444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:43.035391    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:43.053444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:43.036852    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:43:45.629018 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:43:45.628965    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:43:50.138191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:50.130097    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"]
Oct 30 19:43:50.138191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:50.130426    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w" podUID="d30e03c9-c264-4b28-96de-720ac0cc24ba" containerName="istio-proxy" containerID="cri-o://c9ba90746ea8f06bab9e3b971ebb7e5a87022d6ea91ca9289a99f550c6752951" gracePeriod=30
Oct 30 19:43:55.548211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.546416    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:43:55.592567 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.592530    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj"]
Oct 30 19:43:55.594003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.593903    4524 cpu_manager.go:414] "containerMap: removing container" podUID="d30e03c9-c264-4b28-96de-720ac0cc24ba" containerName="istio-proxy"
Oct 30 19:43:55.594003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.593934    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d30e03c9-c264-4b28-96de-720ac0cc24ba" containerName="istio-proxy"
Oct 30 19:43:55.594464 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.594440    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="d30e03c9-c264-4b28-96de-720ac0cc24ba" containerName="istio-proxy"
Oct 30 19:43:55.599478 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.599452    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj"
Oct 30 19:43:55.607064 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.603399    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-7aa217-dockercfg-wv8jw\""
Oct 30 19:43:55.607064 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.603520    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2"]
Oct 30 19:43:55.611565 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.609433    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2"
Oct 30 19:43:55.611565 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.610484    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-7aa217\""
Oct 30 19:43:55.637590 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.637217    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj"]
Oct 30 19:43:55.644953 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.644923    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2"]
Oct 30 19:43:55.658651 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.658329    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"]
Oct 30 19:43:55.673281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.673252    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"]
Oct 30 19:43:55.692435 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.692338    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-data\") pod \"d30e03c9-c264-4b28-96de-720ac0cc24ba\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") "
Oct 30 19:43:55.692578 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.692449    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-token\" (UniqueName: \"kubernetes.io/projected/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-token\") pod \"d30e03c9-c264-4b28-96de-720ac0cc24ba\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") "
Oct 30 19:43:55.692578 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.692496    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d30e03c9-c264-4b28-96de-720ac0cc24ba-kube-api-access-pmmg7\" (UniqueName: \"kubernetes.io/projected/d30e03c9-c264-4b28-96de-720ac0cc24ba-kube-api-access-pmmg7\") pod \"d30e03c9-c264-4b28-96de-720ac0cc24ba\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") "
Oct 30 19:43:55.692578 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.692529    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-envoy\") pod \"d30e03c9-c264-4b28-96de-720ac0cc24ba\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") "
Oct 30 19:43:55.692728 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.692590    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-podinfo\") pod \"d30e03c9-c264-4b28-96de-720ac0cc24ba\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") "
Oct 30 19:43:55.692728 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.692704    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-workload-certs\") pod \"d30e03c9-c264-4b28-96de-720ac0cc24ba\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") "
Oct 30 19:43:55.692867 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.692850    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-workload-socket\") pod \"d30e03c9-c264-4b28-96de-720ac0cc24ba\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") "
Oct 30 19:43:55.693192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.692921    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d30e03c9-c264-4b28-96de-720ac0cc24ba-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d30e03c9-c264-4b28-96de-720ac0cc24ba-istiod-ca-cert\") pod \"d30e03c9-c264-4b28-96de-720ac0cc24ba\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") "
Oct 30 19:43:55.693192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.693020    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-credential-socket\") pod \"d30e03c9-c264-4b28-96de-720ac0cc24ba\" (UID: \"d30e03c9-c264-4b28-96de-720ac0cc24ba\") "
Oct 30 19:43:55.693381 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.693347    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ffe966b-ae60-467e-9c3d-6600bc846a6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2\" (UID: \"6ffe966b-ae60-467e-9c3d-6600bc846a6b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2"
Oct 30 19:43:55.693688 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.693477    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-workload-socket" pod "d30e03c9-c264-4b28-96de-720ac0cc24ba" (UID: "d30e03c9-c264-4b28-96de-720ac0cc24ba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:55.693688 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.693489    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf77p\" (UniqueName: \"kubernetes.io/projected/ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4-kube-api-access-wf77p\") pod \"payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj\" (UID: \"ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj"
Oct 30 19:43:55.693688 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.693614    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5fkw\" (UniqueName: \"kubernetes.io/projected/6ffe966b-ae60-467e-9c3d-6600bc846a6b-kube-api-access-b5fkw\") pod \"payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2\" (UID: \"6ffe966b-ae60-467e-9c3d-6600bc846a6b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2"
Oct 30 19:43:55.693916 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.693720    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-credential-socket" pod "d30e03c9-c264-4b28-96de-720ac0cc24ba" (UID: "d30e03c9-c264-4b28-96de-720ac0cc24ba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:55.693916 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.693880    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-workload-certs" pod "d30e03c9-c264-4b28-96de-720ac0cc24ba" (UID: "d30e03c9-c264-4b28-96de-720ac0cc24ba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:55.694546 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.694123    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d30e03c9-c264-4b28-96de-720ac0cc24ba-istiod-ca-cert" pod "d30e03c9-c264-4b28-96de-720ac0cc24ba" (UID: "d30e03c9-c264-4b28-96de-720ac0cc24ba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:55.694546 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.694296    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj\" (UID: \"ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj"
Oct 30 19:43:55.694546 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.694355    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-data" pod "d30e03c9-c264-4b28-96de-720ac0cc24ba" (UID: "d30e03c9-c264-4b28-96de-720ac0cc24ba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:55.694546 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.694438    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:55.694546 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.694459    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:55.694546 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.694474    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d30e03c9-c264-4b28-96de-720ac0cc24ba-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:55.694546 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.694488    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:55.695920 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.695887    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d30e03c9-c264-4b28-96de-720ac0cc24ba-kube-api-access-pmmg7" pod "d30e03c9-c264-4b28-96de-720ac0cc24ba" (UID: "d30e03c9-c264-4b28-96de-720ac0cc24ba"). InnerVolumeSpecName "kube-api-access-pmmg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:55.696924 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.696899    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-podinfo" pod "d30e03c9-c264-4b28-96de-720ac0cc24ba" (UID: "d30e03c9-c264-4b28-96de-720ac0cc24ba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:55.698135 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.698110    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-token" pod "d30e03c9-c264-4b28-96de-720ac0cc24ba" (UID: "d30e03c9-c264-4b28-96de-720ac0cc24ba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:55.698264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.698154    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-envoy" pod "d30e03c9-c264-4b28-96de-720ac0cc24ba" (UID: "d30e03c9-c264-4b28-96de-720ac0cc24ba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:55.705831 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.705775    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-e50648-dockercfg-9vfgq\""
Oct 30 19:43:55.710613 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.709466    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-e50648\""
Oct 30 19:43:55.796522 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.796477    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ffe966b-ae60-467e-9c3d-6600bc846a6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2\" (UID: \"6ffe966b-ae60-467e-9c3d-6600bc846a6b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2"
Oct 30 19:43:55.796729 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.796632    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf77p\" (UniqueName: \"kubernetes.io/projected/ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4-kube-api-access-wf77p\") pod \"payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj\" (UID: \"ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj"
Oct 30 19:43:55.796729 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.796701    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5fkw\" (UniqueName: \"kubernetes.io/projected/6ffe966b-ae60-467e-9c3d-6600bc846a6b-kube-api-access-b5fkw\") pod \"payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2\" (UID: \"6ffe966b-ae60-467e-9c3d-6600bc846a6b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2"
Oct 30 19:43:55.796871 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.796853    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj\" (UID: \"ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj"
Oct 30 19:43:55.797040 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.797023    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:55.797111 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.797044    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:55.797111 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.797059    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmmg7\" (UniqueName: \"kubernetes.io/projected/d30e03c9-c264-4b28-96de-720ac0cc24ba-kube-api-access-pmmg7\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:55.797111 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.797074    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:55.797111 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.797088    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d30e03c9-c264-4b28-96de-720ac0cc24ba-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:55.797995 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.797968    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj\" (UID: \"ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj"
Oct 30 19:43:55.798720 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.798695    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ffe966b-ae60-467e-9c3d-6600bc846a6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2\" (UID: \"6ffe966b-ae60-467e-9c3d-6600bc846a6b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2"
Oct 30 19:43:55.810897 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.810876    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf77p\" (UniqueName: \"kubernetes.io/projected/ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4-kube-api-access-wf77p\") pod \"payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj\" (UID: \"ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj"
Oct 30 19:43:55.811000 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.810979    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5fkw\" (UniqueName: \"kubernetes.io/projected/6ffe966b-ae60-467e-9c3d-6600bc846a6b-kube-api-access-b5fkw\") pod \"payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2\" (UID: \"6ffe966b-ae60-467e-9c3d-6600bc846a6b\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2"
Oct 30 19:43:55.950802 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.950724    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj"
Oct 30 19:43:55.963946 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:55.963916    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2"
Oct 30 19:43:56.228583 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:56.228494    4524 generic.go:356] "Generic (PLEG): container finished" podID="d30e03c9-c264-4b28-96de-720ac0cc24ba" containerID="c9ba90746ea8f06bab9e3b971ebb7e5a87022d6ea91ca9289a99f550c6752951" exitCode=0
Oct 30 19:43:56.228583 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:56.228560    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"
Oct 30 19:43:56.228583 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:56.228572    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w" event={"ID":"d30e03c9-c264-4b28-96de-720ac0cc24ba","Type":"ContainerDied","Data":"c9ba90746ea8f06bab9e3b971ebb7e5a87022d6ea91ca9289a99f550c6752951"}
Oct 30 19:43:56.228831 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:56.228627    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w" event={"ID":"d30e03c9-c264-4b28-96de-720ac0cc24ba","Type":"ContainerDied","Data":"0e2f2224991cd563f2e17c50b9316bf93481e3f677460f72ae15ced67072241b"}
Oct 30 19:43:56.228831 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:56.228650    4524 scope.go:122] "RemoveContainer" containerID="c9ba90746ea8f06bab9e3b971ebb7e5a87022d6ea91ca9289a99f550c6752951"
Oct 30 19:43:56.240086 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:56.240070    4524 scope.go:122] "RemoveContainer" containerID="c9ba90746ea8f06bab9e3b971ebb7e5a87022d6ea91ca9289a99f550c6752951"
Oct 30 19:43:56.240356 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:43:56.240337    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9ba90746ea8f06bab9e3b971ebb7e5a87022d6ea91ca9289a99f550c6752951\": container with ID starting with c9ba90746ea8f06bab9e3b971ebb7e5a87022d6ea91ca9289a99f550c6752951 not found: ID does not exist" containerID="c9ba90746ea8f06bab9e3b971ebb7e5a87022d6ea91ca9289a99f550c6752951"
Oct 30 19:43:56.240423 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:56.240363    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9ba90746ea8f06bab9e3b971ebb7e5a87022d6ea91ca9289a99f550c6752951"} err="failed to get container status \"c9ba90746ea8f06bab9e3b971ebb7e5a87022d6ea91ca9289a99f550c6752951\": rpc error: code = NotFound desc = could not find container \"c9ba90746ea8f06bab9e3b971ebb7e5a87022d6ea91ca9289a99f550c6752951\": container with ID starting with c9ba90746ea8f06bab9e3b971ebb7e5a87022d6ea91ca9289a99f550c6752951 not found: ID does not exist"
Oct 30 19:43:56.254694 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:56.254670    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"]
Oct 30 19:43:56.259233 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:56.259211    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-0a6e66-openshift-default-68c7bb8bdf-x9p6w"]
Oct 30 19:43:56.376194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:56.369699    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj"]
Oct 30 19:43:56.399132 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:56.399099    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d30e03c9-c264-4b28-96de-720ac0cc24ba" path="/var/lib/kubelet/pods/d30e03c9-c264-4b28-96de-720ac0cc24ba/volumes"
Oct 30 19:43:56.402925 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:56.402900    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2"]
Oct 30 19:43:56.409600 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:43:56.409572    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ffe966b_ae60_467e_9c3d_6600bc846a6b.slice/crio-58f7cbd442de56d8957fdfaf62d7e6404b71f3c4950f1a73dde2858281d8ea93 WatchSource:0}: Error finding container 58f7cbd442de56d8957fdfaf62d7e6404b71f3c4950f1a73dde2858281d8ea93: Status 404 returned error can't find the container with id 58f7cbd442de56d8957fdfaf62d7e6404b71f3c4950f1a73dde2858281d8ea93
Oct 30 19:43:57.246257 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:57.246217    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj" event={"ID":"ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4","Type":"ContainerStarted","Data":"aba3b344a54a8264039bc599981f0d9f00a5bae3849a2d1f632917839811a8a6"}
Oct 30 19:43:57.246722 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:57.246285    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj" event={"ID":"ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4","Type":"ContainerStarted","Data":"6a5869be1dfe1fadca2b31389a4e20ac89bbde69c53670063c272693ba9b6d36"}
Oct 30 19:43:57.247376 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:57.247353    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj"
Oct 30 19:43:57.249231 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:57.249201    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2" event={"ID":"6ffe966b-ae60-467e-9c3d-6600bc846a6b","Type":"ContainerStarted","Data":"58f7cbd442de56d8957fdfaf62d7e6404b71f3c4950f1a73dde2858281d8ea93"}
Oct 30 19:43:57.265820 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:57.265765    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj" podStartSLOduration=12.265747957 podStartE2EDuration="12.265747957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:57.264023616 +0000 UTC m=+2269.005014709" watchObservedRunningTime="2026-10-30 19:43:57.265747957 +0000 UTC m=+2269.006739036"
Oct 30 19:43:59.269848 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:59.269805    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2" event={"ID":"6ffe966b-ae60-467e-9c3d-6600bc846a6b","Type":"ContainerStarted","Data":"1227c1421eefe6f55aa3247de691c308ac184b0d46b02d46445b43796aefa76f"}
Oct 30 19:43:59.270427 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:59.269898    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2"
Oct 30 19:43:59.425194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:43:59.399008    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2" podStartSLOduration=12.377057822 podStartE2EDuration="14.398985021s" totalImagesPullingTime="2.021927199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:43:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:43:59.287541593 +0000 UTC m=+2271.028532687" watchObservedRunningTime="2026-10-30 19:43:59.398985021 +0000 UTC m=+2271.139976099"
Oct 30 19:44:01.100452 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:01.100374    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c"]
Oct 30 19:44:06.101382 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:06.101336    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c" podUID="4bac4098-9dc9-4305-b0ef-286d82cae682" containerName="payload-pre-processing" containerID="cri-o://152542d5a417fc063afb1acfbd5384dd4171ee535adedcde56e9254f330d32d1" gracePeriod=25
Oct 30 19:44:09.273836 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:09.273798    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj"
Oct 30 19:44:10.283517 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:10.283484    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2"
Oct 30 19:44:10.690958 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:10.690911    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8" podUID="59e55265-d446-483c-992c-12134f8ec451" containerName="main" containerID="cri-o://da1de6ac3906f541933e274167ba7c78178d56fe9e84419544dfdb7365aecb89" gracePeriod=45
Oct 30 19:44:10.704011 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:10.703975    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j" podUID="24fbbf1e-3438-4436-b4cd-9890fe28676a" containerName="main" containerID="cri-o://6bbfebe44145e43886a4a87389b690a9fdb123417773335949ed9932b46cabbe" gracePeriod=45
Oct 30 19:44:10.713865 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:10.713827    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8" podUID="59e55265-d446-483c-992c-12134f8ec451" containerName="main" probeResult="failure" output="Get \"https://10.133.0.101:8000/ready\": dial tcp 10.133.0.101:8000: connect: connection refused"
Oct 30 19:44:10.830205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:10.827553    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j" podUID="24fbbf1e-3438-4436-b4cd-9890fe28676a" containerName="main" probeResult="failure" output="Get \"https://10.133.0.98:8000/ready\": dial tcp 10.133.0.98:8000: connect: connection refused"
Oct 30 19:44:11.063538 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.063087    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:44:11.103747 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.103718    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:44:11.224002 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.223927    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/24fbbf1e-3438-4436-b4cd-9890fe28676a-kube-api-access-qttsl\" (UniqueName: \"kubernetes.io/projected/24fbbf1e-3438-4436-b4cd-9890fe28676a-kube-api-access-qttsl\") pod \"24fbbf1e-3438-4436-b4cd-9890fe28676a\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") "
Oct 30 19:44:11.224002 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.223973    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-home\" (UniqueName: \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-home\") pod \"24fbbf1e-3438-4436-b4cd-9890fe28676a\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") "
Oct 30 19:44:11.224266 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.224025    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-home\" (UniqueName: \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-home\") pod \"59e55265-d446-483c-992c-12134f8ec451\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") "
Oct 30 19:44:11.224266 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.224072    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-model-cache\") pod \"24fbbf1e-3438-4436-b4cd-9890fe28676a\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") "
Oct 30 19:44:11.224266 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.224130    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/24fbbf1e-3438-4436-b4cd-9890fe28676a-tls-certs\" (UniqueName: \"kubernetes.io/secret/24fbbf1e-3438-4436-b4cd-9890fe28676a-tls-certs\") pod \"24fbbf1e-3438-4436-b4cd-9890fe28676a\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") "
Oct 30 19:44:11.224434 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.224275    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59e55265-d446-483c-992c-12134f8ec451-kube-api-access-tsrxv\" (UniqueName: \"kubernetes.io/projected/59e55265-d446-483c-992c-12134f8ec451-kube-api-access-tsrxv\") pod \"59e55265-d446-483c-992c-12134f8ec451\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") "
Oct 30 19:44:11.224434 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.224370    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-model-cache" pod "24fbbf1e-3438-4436-b4cd-9890fe28676a" (UID: "24fbbf1e-3438-4436-b4cd-9890fe28676a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:11.224434 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.224382    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-home" pod "24fbbf1e-3438-4436-b4cd-9890fe28676a" (UID: "24fbbf1e-3438-4436-b4cd-9890fe28676a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:11.224434 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.224398    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-home" pod "59e55265-d446-483c-992c-12134f8ec451" (UID: "59e55265-d446-483c-992c-12134f8ec451"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:11.224434 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.224423    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-tmp-dir\") pod \"24fbbf1e-3438-4436-b4cd-9890fe28676a\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") "
Oct 30 19:44:11.224670 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.224500    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/59e55265-d446-483c-992c-12134f8ec451-tls-certs\" (UniqueName: \"kubernetes.io/secret/59e55265-d446-483c-992c-12134f8ec451-tls-certs\") pod \"59e55265-d446-483c-992c-12134f8ec451\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") "
Oct 30 19:44:11.224670 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.224563    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-tmp-dir\") pod \"59e55265-d446-483c-992c-12134f8ec451\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") "
Oct 30 19:44:11.224670 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.224632    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-dshm\" (UniqueName: \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-dshm\") pod \"59e55265-d446-483c-992c-12134f8ec451\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") "
Oct 30 19:44:11.224836 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.224676    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-model-cache\") pod \"59e55265-d446-483c-992c-12134f8ec451\" (UID: \"59e55265-d446-483c-992c-12134f8ec451\") "
Oct 30 19:44:11.224836 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.224706    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-dshm\") pod \"24fbbf1e-3438-4436-b4cd-9890fe28676a\" (UID: \"24fbbf1e-3438-4436-b4cd-9890fe28676a\") "
Oct 30 19:44:11.224836 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.224718    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-tmp-dir" pod "24fbbf1e-3438-4436-b4cd-9890fe28676a" (UID: "24fbbf1e-3438-4436-b4cd-9890fe28676a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:11.225022 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.224961    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-tmp-dir" pod "59e55265-d446-483c-992c-12134f8ec451" (UID: "59e55265-d446-483c-992c-12134f8ec451"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:11.225022 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.224997    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-model-cache" pod "59e55265-d446-483c-992c-12134f8ec451" (UID: "59e55265-d446-483c-992c-12134f8ec451"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:11.225590 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.225567    4524 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-tmp-dir\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:11.225590 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.225593    4524 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-tmp-dir\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:11.225765 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.225603    4524 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-model-cache\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:11.225765 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.225613    4524 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-home\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:11.225765 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.225621    4524 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-home\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:11.225765 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.225629    4524 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-model-cache\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:11.226930 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.226906    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24fbbf1e-3438-4436-b4cd-9890fe28676a-tls-certs" pod "24fbbf1e-3438-4436-b4cd-9890fe28676a" (UID: "24fbbf1e-3438-4436-b4cd-9890fe28676a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:44:11.227236 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.227212    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24fbbf1e-3438-4436-b4cd-9890fe28676a-kube-api-access-qttsl" pod "24fbbf1e-3438-4436-b4cd-9890fe28676a" (UID: "24fbbf1e-3438-4436-b4cd-9890fe28676a"). InnerVolumeSpecName "kube-api-access-qttsl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:11.227236 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.227221    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59e55265-d446-483c-992c-12134f8ec451-tls-certs" pod "59e55265-d446-483c-992c-12134f8ec451" (UID: "59e55265-d446-483c-992c-12134f8ec451"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:44:11.227538 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.227517    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-dshm" pod "59e55265-d446-483c-992c-12134f8ec451" (UID: "59e55265-d446-483c-992c-12134f8ec451"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:11.227621 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.227603    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-dshm" pod "24fbbf1e-3438-4436-b4cd-9890fe28676a" (UID: "24fbbf1e-3438-4436-b4cd-9890fe28676a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:11.227621 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.227610    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59e55265-d446-483c-992c-12134f8ec451-kube-api-access-tsrxv" pod "59e55265-d446-483c-992c-12134f8ec451" (UID: "59e55265-d446-483c-992c-12134f8ec451"). InnerVolumeSpecName "kube-api-access-tsrxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:11.327332 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.327293    4524 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/59e55265-d446-483c-992c-12134f8ec451-tls-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:11.327332 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.327322    4524 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/59e55265-d446-483c-992c-12134f8ec451-dshm\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:11.327743 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.327334    4524 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24fbbf1e-3438-4436-b4cd-9890fe28676a-dshm\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:11.327743 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.327355    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qttsl\" (UniqueName: \"kubernetes.io/projected/24fbbf1e-3438-4436-b4cd-9890fe28676a-kube-api-access-qttsl\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:11.327743 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.327367    4524 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24fbbf1e-3438-4436-b4cd-9890fe28676a-tls-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:11.327743 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.327379    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsrxv\" (UniqueName: \"kubernetes.io/projected/59e55265-d446-483c-992c-12134f8ec451-kube-api-access-tsrxv\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:11.388198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.388128    4524 generic.go:356] "Generic (PLEG): container finished" podID="24fbbf1e-3438-4436-b4cd-9890fe28676a" containerID="6bbfebe44145e43886a4a87389b690a9fdb123417773335949ed9932b46cabbe" exitCode=0
Oct 30 19:44:11.388473 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.388199    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j" event={"ID":"24fbbf1e-3438-4436-b4cd-9890fe28676a","Type":"ContainerDied","Data":"6bbfebe44145e43886a4a87389b690a9fdb123417773335949ed9932b46cabbe"}
Oct 30 19:44:11.388473 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.388238    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j" event={"ID":"24fbbf1e-3438-4436-b4cd-9890fe28676a","Type":"ContainerDied","Data":"c39cb6422f0326b587cadedcb58740fdd461424c8c1e31d84176cfc8007f5c35"}
Oct 30 19:44:11.388473 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.388260    4524 scope.go:122] "RemoveContainer" containerID="6bbfebe44145e43886a4a87389b690a9fdb123417773335949ed9932b46cabbe"
Oct 30 19:44:11.388473 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.388395    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"
Oct 30 19:44:11.390632 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.390603    4524 generic.go:356] "Generic (PLEG): container finished" podID="59e55265-d446-483c-992c-12134f8ec451" containerID="da1de6ac3906f541933e274167ba7c78178d56fe9e84419544dfdb7365aecb89" exitCode=0
Oct 30 19:44:11.390767 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.390741    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8" event={"ID":"59e55265-d446-483c-992c-12134f8ec451","Type":"ContainerDied","Data":"da1de6ac3906f541933e274167ba7c78178d56fe9e84419544dfdb7365aecb89"}
Oct 30 19:44:11.390865 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.390769    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8" event={"ID":"59e55265-d446-483c-992c-12134f8ec451","Type":"ContainerDied","Data":"527914c16e5358d06d990f3e91f6209c2d628ddf39b64021e514aaf2d3a7d1a8"}
Oct 30 19:44:11.390865 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.390821    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"
Oct 30 19:44:11.404656 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.404626    4524 scope.go:122] "RemoveContainer" containerID="6bbfebe44145e43886a4a87389b690a9fdb123417773335949ed9932b46cabbe"
Oct 30 19:44:11.404949 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:44:11.404920    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bbfebe44145e43886a4a87389b690a9fdb123417773335949ed9932b46cabbe\": container with ID starting with 6bbfebe44145e43886a4a87389b690a9fdb123417773335949ed9932b46cabbe not found: ID does not exist" containerID="6bbfebe44145e43886a4a87389b690a9fdb123417773335949ed9932b46cabbe"
Oct 30 19:44:11.405060 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.404958    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bbfebe44145e43886a4a87389b690a9fdb123417773335949ed9932b46cabbe"} err="failed to get container status \"6bbfebe44145e43886a4a87389b690a9fdb123417773335949ed9932b46cabbe\": rpc error: code = NotFound desc = could not find container \"6bbfebe44145e43886a4a87389b690a9fdb123417773335949ed9932b46cabbe\": container with ID starting with 6bbfebe44145e43886a4a87389b690a9fdb123417773335949ed9932b46cabbe not found: ID does not exist"
Oct 30 19:44:11.405060 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.404982    4524 scope.go:122] "RemoveContainer" containerID="da1de6ac3906f541933e274167ba7c78178d56fe9e84419544dfdb7365aecb89"
Oct 30 19:44:11.417749 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.417731    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"]
Oct 30 19:44:11.420478 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.420463    4524 scope.go:122] "RemoveContainer" containerID="da1de6ac3906f541933e274167ba7c78178d56fe9e84419544dfdb7365aecb89"
Oct 30 19:44:11.420717 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:44:11.420691    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da1de6ac3906f541933e274167ba7c78178d56fe9e84419544dfdb7365aecb89\": container with ID starting with da1de6ac3906f541933e274167ba7c78178d56fe9e84419544dfdb7365aecb89 not found: ID does not exist" containerID="da1de6ac3906f541933e274167ba7c78178d56fe9e84419544dfdb7365aecb89"
Oct 30 19:44:11.420806 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.420720    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da1de6ac3906f541933e274167ba7c78178d56fe9e84419544dfdb7365aecb89"} err="failed to get container status \"da1de6ac3906f541933e274167ba7c78178d56fe9e84419544dfdb7365aecb89\": rpc error: code = NotFound desc = could not find container \"da1de6ac3906f541933e274167ba7c78178d56fe9e84419544dfdb7365aecb89\": container with ID starting with da1de6ac3906f541933e274167ba7c78178d56fe9e84419544dfdb7365aecb89 not found: ID does not exist"
Oct 30 19:44:11.423108 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.423088    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-bce71f/distinct-w6-bce71f-kserve-5f5b44dbc-q5r9j"]
Oct 30 19:44:11.430210 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.430145    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"]
Oct 30 19:44:11.440563 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:11.440281    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-bce71f/distinct-2-w6-bce71f-kserve-7cc5d6d78d-5jcp8"]
Oct 30 19:44:12.422193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:12.420913    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="24fbbf1e-3438-4436-b4cd-9890fe28676a" path="/var/lib/kubelet/pods/24fbbf1e-3438-4436-b4cd-9890fe28676a/volumes"
Oct 30 19:44:12.424244 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:12.424215    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="59e55265-d446-483c-992c-12134f8ec451" path="/var/lib/kubelet/pods/59e55265-d446-483c-992c-12134f8ec451/volumes"
Oct 30 19:44:12.504608 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:12.504566    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c" podUID="4bac4098-9dc9-4305-b0ef-286d82cae682" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:44:15.626853 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:44:15.626813    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:44:22.535990 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:22.535889    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c" podUID="4bac4098-9dc9-4305-b0ef-286d82cae682" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:44:26.459989 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:26.459958    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c"
Oct 30 19:44:26.532974 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:26.532937    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4bac4098-9dc9-4305-b0ef-286d82cae682-kube-api-access-whq2r\" (UniqueName: \"kubernetes.io/projected/4bac4098-9dc9-4305-b0ef-286d82cae682-kube-api-access-whq2r\") pod \"4bac4098-9dc9-4305-b0ef-286d82cae682\" (UID: \"4bac4098-9dc9-4305-b0ef-286d82cae682\") "
Oct 30 19:44:26.533462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:26.533440    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4bac4098-9dc9-4305-b0ef-286d82cae682-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4bac4098-9dc9-4305-b0ef-286d82cae682-plugins-config-volume\") pod \"4bac4098-9dc9-4305-b0ef-286d82cae682\" (UID: \"4bac4098-9dc9-4305-b0ef-286d82cae682\") "
Oct 30 19:44:26.534931 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:26.534900    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bac4098-9dc9-4305-b0ef-286d82cae682-plugins-config-volume" pod "4bac4098-9dc9-4305-b0ef-286d82cae682" (UID: "4bac4098-9dc9-4305-b0ef-286d82cae682"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:44:26.537059 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:26.537034    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bac4098-9dc9-4305-b0ef-286d82cae682-kube-api-access-whq2r" pod "4bac4098-9dc9-4305-b0ef-286d82cae682" (UID: "4bac4098-9dc9-4305-b0ef-286d82cae682"). InnerVolumeSpecName "kube-api-access-whq2r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:26.674244 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:26.664730    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whq2r\" (UniqueName: \"kubernetes.io/projected/4bac4098-9dc9-4305-b0ef-286d82cae682-kube-api-access-whq2r\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:26.674244 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:26.664771    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4bac4098-9dc9-4305-b0ef-286d82cae682-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:26.833327 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:26.833291    4524 generic.go:356] "Generic (PLEG): container finished" podID="4bac4098-9dc9-4305-b0ef-286d82cae682" containerID="152542d5a417fc063afb1acfbd5384dd4171ee535adedcde56e9254f330d32d1" exitCode=0
Oct 30 19:44:26.833541 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:26.833336    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c"
Oct 30 19:44:26.833541 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:26.833343    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c" event={"ID":"4bac4098-9dc9-4305-b0ef-286d82cae682","Type":"ContainerDied","Data":"152542d5a417fc063afb1acfbd5384dd4171ee535adedcde56e9254f330d32d1"}
Oct 30 19:44:26.833541 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:26.833396    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c" event={"ID":"4bac4098-9dc9-4305-b0ef-286d82cae682","Type":"ContainerDied","Data":"3d482526290526ca442ae22519a0e387c9c282ff70a3e165c5705a379abede8f"}
Oct 30 19:44:26.833541 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:26.833425    4524 scope.go:122] "RemoveContainer" containerID="152542d5a417fc063afb1acfbd5384dd4171ee535adedcde56e9254f330d32d1"
Oct 30 19:44:26.844686 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:26.844665    4524 scope.go:122] "RemoveContainer" containerID="152542d5a417fc063afb1acfbd5384dd4171ee535adedcde56e9254f330d32d1"
Oct 30 19:44:26.844951 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:44:26.844928    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"152542d5a417fc063afb1acfbd5384dd4171ee535adedcde56e9254f330d32d1\": container with ID starting with 152542d5a417fc063afb1acfbd5384dd4171ee535adedcde56e9254f330d32d1 not found: ID does not exist" containerID="152542d5a417fc063afb1acfbd5384dd4171ee535adedcde56e9254f330d32d1"
Oct 30 19:44:26.845038 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:26.844960    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"152542d5a417fc063afb1acfbd5384dd4171ee535adedcde56e9254f330d32d1"} err="failed to get container status \"152542d5a417fc063afb1acfbd5384dd4171ee535adedcde56e9254f330d32d1\": rpc error: code = NotFound desc = could not find container \"152542d5a417fc063afb1acfbd5384dd4171ee535adedcde56e9254f330d32d1\": container with ID starting with 152542d5a417fc063afb1acfbd5384dd4171ee535adedcde56e9254f330d32d1 not found: ID does not exist"
Oct 30 19:44:26.859088 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:26.859065    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c"]
Oct 30 19:44:26.863569 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:26.863545    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w6-bce71f-6fd77677dc-d7s5c"]
Oct 30 19:44:28.396987 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:28.396943    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4bac4098-9dc9-4305-b0ef-286d82cae682" path="/var/lib/kubelet/pods/4bac4098-9dc9-4305-b0ef-286d82cae682/volumes"
Oct 30 19:44:33.932706 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:33.932673    4524 generic.go:356] "Generic (PLEG): container finished" podID="881f121f-ca0c-4a99-a9d2-d0cca6b54a48" containerID="905aaceacb19f21070b83bc9cfd84e72fc956abb656dc5f788f154c479933cd5" exitCode=0
Oct 30 19:44:33.933153 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:33.932724    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql" event={"ID":"881f121f-ca0c-4a99-a9d2-d0cca6b54a48","Type":"ContainerDied","Data":"905aaceacb19f21070b83bc9cfd84e72fc956abb656dc5f788f154c479933cd5"}
Oct 30 19:44:33.933236 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:33.933164    4524 scope.go:122] "RemoveContainer" containerID="905aaceacb19f21070b83bc9cfd84e72fc956abb656dc5f788f154c479933cd5"
Oct 30 19:44:33.935127 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:33.935098    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:44:33.935254 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:33.935210    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:44:33.935305 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:33.935270    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:44:35.003198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:35.000011    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql" event={"ID":"881f121f-ca0c-4a99-a9d2-d0cca6b54a48","Type":"ContainerStarted","Data":"7758f773061f729e5d4ad5f10c00db3a296e448cb773107ceea5b7f2674ab3f7"}
Oct 30 19:44:35.403563 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:35.403532    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:44:35.403937 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:35.403915    4524 patch_prober.go:29] interesting pod/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:44:35.404112 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:35.404088    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql" podUID="881f121f-ca0c-4a99-a9d2-d0cca6b54a48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 19:44:36.403335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:36.403293    4524 patch_prober.go:29] interesting pod/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:44:36.403776 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:36.403367    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql" podUID="881f121f-ca0c-4a99-a9d2-d0cca6b54a48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 19:44:37.409196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:37.408396    4524 patch_prober.go:29] interesting pod/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:44:37.409196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:37.408454    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql" podUID="881f121f-ca0c-4a99-a9d2-d0cca6b54a48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 19:44:38.407195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:38.405630    4524 patch_prober.go:29] interesting pod/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:44:38.407195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:38.405709    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql" podUID="881f121f-ca0c-4a99-a9d2-d0cca6b54a48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 19:44:39.421586 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:39.421549    4524 patch_prober.go:29] interesting pod/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:44:39.422311 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:39.422275    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql" podUID="881f121f-ca0c-4a99-a9d2-d0cca6b54a48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 19:44:40.311636 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.311595    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn"]
Oct 30 19:44:40.313528 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.313501    4524 cpu_manager.go:414] "containerMap: removing container" podUID="24fbbf1e-3438-4436-b4cd-9890fe28676a" containerName="main"
Oct 30 19:44:40.313528 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.313528    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="24fbbf1e-3438-4436-b4cd-9890fe28676a" containerName="main"
Oct 30 19:44:40.313673 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.313623    4524 cpu_manager.go:414] "containerMap: removing container" podUID="4bac4098-9dc9-4305-b0ef-286d82cae682" containerName="payload-pre-processing"
Oct 30 19:44:40.313673 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.313631    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4bac4098-9dc9-4305-b0ef-286d82cae682" containerName="payload-pre-processing"
Oct 30 19:44:40.313673 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.313662    4524 cpu_manager.go:414] "containerMap: removing container" podUID="59e55265-d446-483c-992c-12134f8ec451" containerName="main"
Oct 30 19:44:40.313673 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.313670    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="59e55265-d446-483c-992c-12134f8ec451" containerName="main"
Oct 30 19:44:40.313897 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.313884    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="4bac4098-9dc9-4305-b0ef-286d82cae682" containerName="payload-pre-processing"
Oct 30 19:44:40.313933 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.313905    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="59e55265-d446-483c-992c-12134f8ec451" containerName="main"
Oct 30 19:44:40.313933 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.313921    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="24fbbf1e-3438-4436-b4cd-9890fe28676a" containerName="main"
Oct 30 19:44:40.318596 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.318574    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn"
Oct 30 19:44:40.325589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.325565    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-1a844e\""
Oct 30 19:44:40.325712 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.325652    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-1a844e-dockercfg-4772r\""
Oct 30 19:44:40.327120 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.327098    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn"]
Oct 30 19:44:40.499205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.497743    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfgg8\" (UniqueName: \"kubernetes.io/projected/7f74a4df-72f1-4a7c-a468-76deb45bf8ac-kube-api-access-mfgg8\") pod \"payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn\" (UID: \"7f74a4df-72f1-4a7c-a468-76deb45bf8ac\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn"
Oct 30 19:44:40.499205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.498040    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f74a4df-72f1-4a7c-a468-76deb45bf8ac-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn\" (UID: \"7f74a4df-72f1-4a7c-a468-76deb45bf8ac\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn"
Oct 30 19:44:40.540924 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.540294    4524 patch_prober.go:29] interesting pod/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:44:40.540924 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.540372    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql" podUID="881f121f-ca0c-4a99-a9d2-d0cca6b54a48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 19:44:40.628193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.627519    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f74a4df-72f1-4a7c-a468-76deb45bf8ac-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn\" (UID: \"7f74a4df-72f1-4a7c-a468-76deb45bf8ac\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn"
Oct 30 19:44:40.628193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.627674    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfgg8\" (UniqueName: \"kubernetes.io/projected/7f74a4df-72f1-4a7c-a468-76deb45bf8ac-kube-api-access-mfgg8\") pod \"payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn\" (UID: \"7f74a4df-72f1-4a7c-a468-76deb45bf8ac\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn"
Oct 30 19:44:40.656195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.628982    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f74a4df-72f1-4a7c-a468-76deb45bf8ac-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn\" (UID: \"7f74a4df-72f1-4a7c-a468-76deb45bf8ac\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn"
Oct 30 19:44:40.692535 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.691771    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfgg8\" (UniqueName: \"kubernetes.io/projected/7f74a4df-72f1-4a7c-a468-76deb45bf8ac-kube-api-access-mfgg8\") pod \"payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn\" (UID: \"7f74a4df-72f1-4a7c-a468-76deb45bf8ac\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn"
Oct 30 19:44:40.889252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:40.886463    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn"
Oct 30 19:44:41.404195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:41.403544    4524 patch_prober.go:29] interesting pod/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:44:41.404195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:41.403620    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql" podUID="881f121f-ca0c-4a99-a9d2-d0cca6b54a48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.170:15021/healthz/ready\": dial tcp 10.133.0.170:15021: connect: connection refused"
Oct 30 19:44:41.635408 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:41.635373    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn"]
Oct 30 19:44:42.765300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:42.760823    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:44:42.771339 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:42.768216    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:44:42.794544 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:42.780269    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn" event={"ID":"7f74a4df-72f1-4a7c-a468-76deb45bf8ac","Type":"ContainerStarted","Data":"d0a544627ce45c4561c55d7b5b28443021ff77e457f76641ceab2c5823c450e1"}
Oct 30 19:44:42.794544 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:42.780336    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn" event={"ID":"7f74a4df-72f1-4a7c-a468-76deb45bf8ac","Type":"ContainerStarted","Data":"4e1d27e4d922be4fdf5bfd1fb1b3fb8f4f367608d64c2452f6312ecadae77ded"}
Oct 30 19:44:42.794544 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:42.781234    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn"
Oct 30 19:44:42.795330 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:42.795299    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:44:43.170328 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:43.170263    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn" podStartSLOduration=3.170240477 podStartE2EDuration="3.170240477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:42.895576962 +0000 UTC m=+2314.636568042" watchObservedRunningTime="2026-10-30 19:44:43.170240477 +0000 UTC m=+2314.911231578"
Oct 30 19:44:45.640118 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:44:45.640080    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:44:48.010831 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:48.010794    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"]
Oct 30 19:44:48.011693 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:48.011639    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj" podUID="77f56e59-c42b-4a38-a6c8-85f903e73922" containerName="istio-proxy" containerID="cri-o://e3289a960ec230768eb64b2f42165e810fac1ee7b78a83c256020e17d7c73943" gracePeriod=30
Oct 30 19:44:48.025277 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:48.025247    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"]
Oct 30 19:44:48.025481 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:48.025458    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77" podUID="d898cf4d-a645-494a-a4af-88830acf24ab" containerName="istio-proxy" containerID="cri-o://8eb10a7e60e00259dcee362b163edc8ea0d194016806354006be7861165e914a" gracePeriod=30
Oct 30 19:44:48.760696 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:48.760657    4524 patch_prober.go:29] interesting pod/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:44:48.760896 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:48.760718    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj" podUID="77f56e59-c42b-4a38-a6c8-85f903e73922" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:44:53.489326 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.488811    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:44:53.491418 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.491391    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:44:53.528717 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.528686    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"]
Oct 30 19:44:53.529948 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.529922    4524 cpu_manager.go:414] "containerMap: removing container" podUID="d898cf4d-a645-494a-a4af-88830acf24ab" containerName="istio-proxy"
Oct 30 19:44:53.530103 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.530090    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d898cf4d-a645-494a-a4af-88830acf24ab" containerName="istio-proxy"
Oct 30 19:44:53.530252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.530235    4524 cpu_manager.go:414] "containerMap: removing container" podUID="77f56e59-c42b-4a38-a6c8-85f903e73922" containerName="istio-proxy"
Oct 30 19:44:53.530325 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.530252    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="77f56e59-c42b-4a38-a6c8-85f903e73922" containerName="istio-proxy"
Oct 30 19:44:53.530447 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.530433    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="d898cf4d-a645-494a-a4af-88830acf24ab" containerName="istio-proxy"
Oct 30 19:44:53.530517 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.530454    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="77f56e59-c42b-4a38-a6c8-85f903e73922" containerName="istio-proxy"
Oct 30 19:44:53.534981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.534404    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"
Oct 30 19:44:53.541847 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.541764    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"]
Oct 30 19:44:53.573573 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.570299    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-f5613a-serving-cert\""
Oct 30 19:44:53.573573 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.570349    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"]
Oct 30 19:44:53.580147 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.579390    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"]
Oct 30 19:44:53.580147 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.579508    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.594226 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.593901    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-workload-certs\") pod \"d898cf4d-a645-494a-a4af-88830acf24ab\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") "
Oct 30 19:44:53.594226 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.593954    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/77f56e59-c42b-4a38-a6c8-85f903e73922-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77f56e59-c42b-4a38-a6c8-85f903e73922-istiod-ca-cert\") pod \"77f56e59-c42b-4a38-a6c8-85f903e73922\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") "
Oct 30 19:44:53.594226 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.594007    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-envoy\") pod \"77f56e59-c42b-4a38-a6c8-85f903e73922\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") "
Oct 30 19:44:53.594226 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.594084    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-data\") pod \"77f56e59-c42b-4a38-a6c8-85f903e73922\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") "
Oct 30 19:44:53.594226 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.594143    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d898cf4d-a645-494a-a4af-88830acf24ab-istio-token\" (UniqueName: \"kubernetes.io/projected/d898cf4d-a645-494a-a4af-88830acf24ab-istio-token\") pod \"d898cf4d-a645-494a-a4af-88830acf24ab\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") "
Oct 30 19:44:53.594226 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.594208    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-workload-certs\") pod \"77f56e59-c42b-4a38-a6c8-85f903e73922\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") "
Oct 30 19:44:53.594799 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.594288    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-workload-socket\") pod \"77f56e59-c42b-4a38-a6c8-85f903e73922\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") "
Oct 30 19:44:53.594799 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.594321    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-istio-envoy\") pod \"d898cf4d-a645-494a-a4af-88830acf24ab\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") "
Oct 30 19:44:53.594799 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.594359    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d898cf4d-a645-494a-a4af-88830acf24ab-kube-api-access-52jn9\" (UniqueName: \"kubernetes.io/projected/d898cf4d-a645-494a-a4af-88830acf24ab-kube-api-access-52jn9\") pod \"d898cf4d-a645-494a-a4af-88830acf24ab\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") "
Oct 30 19:44:53.594799 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.594395    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77f56e59-c42b-4a38-a6c8-85f903e73922-kube-api-access-xgb87\" (UniqueName: \"kubernetes.io/projected/77f56e59-c42b-4a38-a6c8-85f903e73922-kube-api-access-xgb87\") pod \"77f56e59-c42b-4a38-a6c8-85f903e73922\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") "
Oct 30 19:44:53.594799 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.594446    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-credential-socket\") pod \"77f56e59-c42b-4a38-a6c8-85f903e73922\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") "
Oct 30 19:44:53.594799 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.594542    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-token\" (UniqueName: \"kubernetes.io/projected/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-token\") pod \"77f56e59-c42b-4a38-a6c8-85f903e73922\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") "
Oct 30 19:44:53.594799 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.594595    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-podinfo\") pod \"77f56e59-c42b-4a38-a6c8-85f903e73922\" (UID: \"77f56e59-c42b-4a38-a6c8-85f903e73922\") "
Oct 30 19:44:53.594799 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.594633    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d898cf4d-a645-494a-a4af-88830acf24ab-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d898cf4d-a645-494a-a4af-88830acf24ab-istiod-ca-cert\") pod \"d898cf4d-a645-494a-a4af-88830acf24ab\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") "
Oct 30 19:44:53.594799 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.594667    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-istio-data\") pod \"d898cf4d-a645-494a-a4af-88830acf24ab\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") "
Oct 30 19:44:53.594799 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.594759    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-credential-socket\") pod \"d898cf4d-a645-494a-a4af-88830acf24ab\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") "
Oct 30 19:44:53.595397 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.594809    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d898cf4d-a645-494a-a4af-88830acf24ab-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d898cf4d-a645-494a-a4af-88830acf24ab-istio-podinfo\") pod \"d898cf4d-a645-494a-a4af-88830acf24ab\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") "
Oct 30 19:44:53.595397 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.594885    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-workload-socket\") pod \"d898cf4d-a645-494a-a4af-88830acf24ab\" (UID: \"d898cf4d-a645-494a-a4af-88830acf24ab\") "
Oct 30 19:44:53.596138 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.596105    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-credential-socket" pod "77f56e59-c42b-4a38-a6c8-85f903e73922" (UID: "77f56e59-c42b-4a38-a6c8-85f903e73922"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:53.596573 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.596450    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-istio-data" pod "d898cf4d-a645-494a-a4af-88830acf24ab" (UID: "d898cf4d-a645-494a-a4af-88830acf24ab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:53.596826 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.596807    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d898cf4d-a645-494a-a4af-88830acf24ab-istiod-ca-cert" pod "d898cf4d-a645-494a-a4af-88830acf24ab" (UID: "d898cf4d-a645-494a-a4af-88830acf24ab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:44:53.597546 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.597515    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-workload-certs" pod "77f56e59-c42b-4a38-a6c8-85f903e73922" (UID: "77f56e59-c42b-4a38-a6c8-85f903e73922"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:53.597998 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.597974    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-data" pod "77f56e59-c42b-4a38-a6c8-85f903e73922" (UID: "77f56e59-c42b-4a38-a6c8-85f903e73922"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:53.598215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.598184    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-workload-certs" pod "d898cf4d-a645-494a-a4af-88830acf24ab" (UID: "d898cf4d-a645-494a-a4af-88830acf24ab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:53.598645 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.598567    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77f56e59-c42b-4a38-a6c8-85f903e73922-istiod-ca-cert" pod "77f56e59-c42b-4a38-a6c8-85f903e73922" (UID: "77f56e59-c42b-4a38-a6c8-85f903e73922"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:44:53.599105 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.599078    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-credential-socket" pod "d898cf4d-a645-494a-a4af-88830acf24ab" (UID: "d898cf4d-a645-494a-a4af-88830acf24ab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:53.607250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.605591    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-envoy" pod "77f56e59-c42b-4a38-a6c8-85f903e73922" (UID: "77f56e59-c42b-4a38-a6c8-85f903e73922"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:53.607250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.605671    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-podinfo" pod "77f56e59-c42b-4a38-a6c8-85f903e73922" (UID: "77f56e59-c42b-4a38-a6c8-85f903e73922"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:44:53.607250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.605875    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-workload-socket" pod "d898cf4d-a645-494a-a4af-88830acf24ab" (UID: "d898cf4d-a645-494a-a4af-88830acf24ab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:53.608344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.607613    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d898cf4d-a645-494a-a4af-88830acf24ab-istio-podinfo" pod "d898cf4d-a645-494a-a4af-88830acf24ab" (UID: "d898cf4d-a645-494a-a4af-88830acf24ab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:44:53.608344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.608194    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:53.608344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.608218    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d898cf4d-a645-494a-a4af-88830acf24ab-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:53.608344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.608232    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:53.608344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.608248    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:53.608344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.608261    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d898cf4d-a645-494a-a4af-88830acf24ab-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:53.608344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.608274    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:53.608344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.608288    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:53.608344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.608301    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/77f56e59-c42b-4a38-a6c8-85f903e73922-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:53.608344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.608318    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:53.608344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.608331    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:53.608344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.608346    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:53.610273 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.608359    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:53.610273 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.608563    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-workload-socket" pod "77f56e59-c42b-4a38-a6c8-85f903e73922" (UID: "77f56e59-c42b-4a38-a6c8-85f903e73922"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:53.619651 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.619618    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d898cf4d-a645-494a-a4af-88830acf24ab-istio-token" pod "d898cf4d-a645-494a-a4af-88830acf24ab" (UID: "d898cf4d-a645-494a-a4af-88830acf24ab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:53.619765 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.619666    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-token" pod "77f56e59-c42b-4a38-a6c8-85f903e73922" (UID: "77f56e59-c42b-4a38-a6c8-85f903e73922"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:53.620710 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.619941    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77f56e59-c42b-4a38-a6c8-85f903e73922-kube-api-access-xgb87" pod "77f56e59-c42b-4a38-a6c8-85f903e73922" (UID: "77f56e59-c42b-4a38-a6c8-85f903e73922"). InnerVolumeSpecName "kube-api-access-xgb87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:53.620710 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.619966    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d898cf4d-a645-494a-a4af-88830acf24ab-kube-api-access-52jn9" pod "d898cf4d-a645-494a-a4af-88830acf24ab" (UID: "d898cf4d-a645-494a-a4af-88830acf24ab"). InnerVolumeSpecName "kube-api-access-52jn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:44:53.622149 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.622130    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-istio-envoy" pod "d898cf4d-a645-494a-a4af-88830acf24ab" (UID: "d898cf4d-a645-494a-a4af-88830acf24ab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:44:53.670585 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.670506    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-6d9f17-openshift-default-dockercfg-b7qln\""
Oct 30 19:44:53.709615 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.709571    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-data\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.709804 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.709630    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-credential-socket\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.709804 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.709752    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e38d835-2f71-40a8-932e-1b6328c9f650-istiod-ca-cert\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.709804 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.709789    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eec1b428-31dc-4bb4-812e-c575b780ea7f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7\" (UID: \"eec1b428-31dc-4bb4-812e-c575b780ea7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"
Oct 30 19:44:53.709945 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.709899    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-podinfo\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.709945 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.709927    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eec1b428-31dc-4bb4-812e-c575b780ea7f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7\" (UID: \"eec1b428-31dc-4bb4-812e-c575b780ea7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"
Oct 30 19:44:53.710079 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.710064    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-workload-certs\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.710222 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.710199    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-envoy\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.710331 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.710315    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-workload-socket\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.710375 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.710358    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-token\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.710456 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.710439    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsfh4\" (UniqueName: \"kubernetes.io/projected/eec1b428-31dc-4bb4-812e-c575b780ea7f-kube-api-access-vsfh4\") pod \"maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7\" (UID: \"eec1b428-31dc-4bb4-812e-c575b780ea7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"
Oct 30 19:44:53.710498 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.710488    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r27lb\" (UniqueName: \"kubernetes.io/projected/0e38d835-2f71-40a8-932e-1b6328c9f650-kube-api-access-r27lb\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.710549 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.710535    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eec1b428-31dc-4bb4-812e-c575b780ea7f-metrics-certs\") pod \"maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7\" (UID: \"eec1b428-31dc-4bb4-812e-c575b780ea7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"
Oct 30 19:44:53.710706 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.710692    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d898cf4d-a645-494a-a4af-88830acf24ab-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:53.710746 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.710713    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/77f56e59-c42b-4a38-a6c8-85f903e73922-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:53.710746 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.710728    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d898cf4d-a645-494a-a4af-88830acf24ab-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:53.710811 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.710743    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52jn9\" (UniqueName: \"kubernetes.io/projected/d898cf4d-a645-494a-a4af-88830acf24ab-kube-api-access-52jn9\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:53.710811 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.710757    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgb87\" (UniqueName: \"kubernetes.io/projected/77f56e59-c42b-4a38-a6c8-85f903e73922-kube-api-access-xgb87\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:53.710811 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.710772    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77f56e59-c42b-4a38-a6c8-85f903e73922-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:44:53.811689 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.811656    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-workload-certs\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.811892 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.811764    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-envoy\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.811892 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.811827    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-workload-socket\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.811892 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.811866    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-token\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.812093 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.811911    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsfh4\" (UniqueName: \"kubernetes.io/projected/eec1b428-31dc-4bb4-812e-c575b780ea7f-kube-api-access-vsfh4\") pod \"maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7\" (UID: \"eec1b428-31dc-4bb4-812e-c575b780ea7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"
Oct 30 19:44:53.812093 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.811957    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r27lb\" (UniqueName: \"kubernetes.io/projected/0e38d835-2f71-40a8-932e-1b6328c9f650-kube-api-access-r27lb\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.812093 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.812005    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eec1b428-31dc-4bb4-812e-c575b780ea7f-metrics-certs\") pod \"maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7\" (UID: \"eec1b428-31dc-4bb4-812e-c575b780ea7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"
Oct 30 19:44:53.812093 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.812086    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-data\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.812325 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.812116    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-credential-socket\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.812325 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.812133    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-workload-certs\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.812325 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.812185    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e38d835-2f71-40a8-932e-1b6328c9f650-istiod-ca-cert\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.812325 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.812213    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eec1b428-31dc-4bb4-812e-c575b780ea7f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7\" (UID: \"eec1b428-31dc-4bb4-812e-c575b780ea7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"
Oct 30 19:44:53.812325 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.812232    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-workload-socket\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.812325 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.812269    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-podinfo\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.812325 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.812292    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eec1b428-31dc-4bb4-812e-c575b780ea7f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7\" (UID: \"eec1b428-31dc-4bb4-812e-c575b780ea7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"
Oct 30 19:44:53.813186 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.813143    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eec1b428-31dc-4bb4-812e-c575b780ea7f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7\" (UID: \"eec1b428-31dc-4bb4-812e-c575b780ea7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"
Oct 30 19:44:53.813356 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.813165    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e38d835-2f71-40a8-932e-1b6328c9f650-istiod-ca-cert\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.813463 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.813436    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-credential-socket\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.813530 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.813220    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-data\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.814566 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.814535    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-envoy\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.815505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.815482    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eec1b428-31dc-4bb4-812e-c575b780ea7f-metrics-certs\") pod \"maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7\" (UID: \"eec1b428-31dc-4bb4-812e-c575b780ea7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"
Oct 30 19:44:53.815625 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.815482    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-podinfo\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.815810 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.815793    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eec1b428-31dc-4bb4-812e-c575b780ea7f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7\" (UID: \"eec1b428-31dc-4bb4-812e-c575b780ea7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"
Oct 30 19:44:53.824131 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.824104    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-token\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.824264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.824129    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsfh4\" (UniqueName: \"kubernetes.io/projected/eec1b428-31dc-4bb4-812e-c575b780ea7f-kube-api-access-vsfh4\") pod \"maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7\" (UID: \"eec1b428-31dc-4bb4-812e-c575b780ea7f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"
Oct 30 19:44:53.824355 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.824336    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r27lb\" (UniqueName: \"kubernetes.io/projected/0e38d835-2f71-40a8-932e-1b6328c9f650-kube-api-access-r27lb\") pod \"e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") " pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.858518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.858475    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"
Oct 30 19:44:53.918779 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.914458    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:53.965350 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.958767    4524 generic.go:356] "Generic (PLEG): container finished" podID="d898cf4d-a645-494a-a4af-88830acf24ab" containerID="8eb10a7e60e00259dcee362b163edc8ea0d194016806354006be7861165e914a" exitCode=0
Oct 30 19:44:53.965350 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.958900    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77" event={"ID":"d898cf4d-a645-494a-a4af-88830acf24ab","Type":"ContainerDied","Data":"8eb10a7e60e00259dcee362b163edc8ea0d194016806354006be7861165e914a"}
Oct 30 19:44:53.965350 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.958957    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77" event={"ID":"d898cf4d-a645-494a-a4af-88830acf24ab","Type":"ContainerDied","Data":"3946a6e07356170ff4edb75dbd9dba30714d5b79017f7ed1aebc66b91270fb1b"}
Oct 30 19:44:53.965350 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.958989    4524 scope.go:122] "RemoveContainer" containerID="8eb10a7e60e00259dcee362b163edc8ea0d194016806354006be7861165e914a"
Oct 30 19:44:53.965350 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.959188    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"
Oct 30 19:44:53.965350 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.962623    4524 generic.go:356] "Generic (PLEG): container finished" podID="77f56e59-c42b-4a38-a6c8-85f903e73922" containerID="e3289a960ec230768eb64b2f42165e810fac1ee7b78a83c256020e17d7c73943" exitCode=0
Oct 30 19:44:53.965350 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.962726    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj" event={"ID":"77f56e59-c42b-4a38-a6c8-85f903e73922","Type":"ContainerDied","Data":"e3289a960ec230768eb64b2f42165e810fac1ee7b78a83c256020e17d7c73943"}
Oct 30 19:44:53.965350 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.962780    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj" event={"ID":"77f56e59-c42b-4a38-a6c8-85f903e73922","Type":"ContainerDied","Data":"9df6157daa8c8761fbfd71db1d6f2fb38360ac0d1d7f62e2c7b4d20ea33dd72e"}
Oct 30 19:44:53.965350 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.962855    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"
Oct 30 19:44:53.974924 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.974904    4524 scope.go:122] "RemoveContainer" containerID="8eb10a7e60e00259dcee362b163edc8ea0d194016806354006be7861165e914a"
Oct 30 19:44:53.975397 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:44:53.975371    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eb10a7e60e00259dcee362b163edc8ea0d194016806354006be7861165e914a\": container with ID starting with 8eb10a7e60e00259dcee362b163edc8ea0d194016806354006be7861165e914a not found: ID does not exist" containerID="8eb10a7e60e00259dcee362b163edc8ea0d194016806354006be7861165e914a"
Oct 30 19:44:53.975514 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.975408    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eb10a7e60e00259dcee362b163edc8ea0d194016806354006be7861165e914a"} err="failed to get container status \"8eb10a7e60e00259dcee362b163edc8ea0d194016806354006be7861165e914a\": rpc error: code = NotFound desc = could not find container \"8eb10a7e60e00259dcee362b163edc8ea0d194016806354006be7861165e914a\": container with ID starting with 8eb10a7e60e00259dcee362b163edc8ea0d194016806354006be7861165e914a not found: ID does not exist"
Oct 30 19:44:53.975514 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:53.975432    4524 scope.go:122] "RemoveContainer" containerID="e3289a960ec230768eb64b2f42165e810fac1ee7b78a83c256020e17d7c73943"
Oct 30 19:44:54.011372 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.011341    4524 scope.go:122] "RemoveContainer" containerID="e3289a960ec230768eb64b2f42165e810fac1ee7b78a83c256020e17d7c73943"
Oct 30 19:44:54.012108 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:44:54.012077    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3289a960ec230768eb64b2f42165e810fac1ee7b78a83c256020e17d7c73943\": container with ID starting with e3289a960ec230768eb64b2f42165e810fac1ee7b78a83c256020e17d7c73943 not found: ID does not exist" containerID="e3289a960ec230768eb64b2f42165e810fac1ee7b78a83c256020e17d7c73943"
Oct 30 19:44:54.012249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.012119    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3289a960ec230768eb64b2f42165e810fac1ee7b78a83c256020e17d7c73943"} err="failed to get container status \"e3289a960ec230768eb64b2f42165e810fac1ee7b78a83c256020e17d7c73943\": rpc error: code = NotFound desc = could not find container \"e3289a960ec230768eb64b2f42165e810fac1ee7b78a83c256020e17d7c73943\": container with ID starting with e3289a960ec230768eb64b2f42165e810fac1ee7b78a83c256020e17d7c73943 not found: ID does not exist"
Oct 30 19:44:54.012249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.012154    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"]
Oct 30 19:44:54.026228 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.019299    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-txp77"]
Oct 30 19:44:54.031355 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.031334    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"]
Oct 30 19:44:54.094706 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.090968    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-bce71f-openshift-default-5d99f47777-w9tdj"]
Oct 30 19:44:54.111051 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.110320    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5"]
Oct 30 19:44:54.112531 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:44:54.112044    4524 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77f56e59_c42b_4a38_a6c8_85f903e73922.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 19:44:54.117576 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.117420    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b5ecb2fa-6w9g5"]
Oct 30 19:44:54.262528 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.262419    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"]
Oct 30 19:44:54.265120 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.265088    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:44:54.265210 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.265153    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:44:54.265210 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.265201    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:44:54.271866 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.271841    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"]
Oct 30 19:44:54.395601 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.395539    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="22b2768d-5577-40ca-b8da-50fea4f50651" path="/var/lib/kubelet/pods/22b2768d-5577-40ca-b8da-50fea4f50651/volumes"
Oct 30 19:44:54.396296 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.396272    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="77f56e59-c42b-4a38-a6c8-85f903e73922" path="/var/lib/kubelet/pods/77f56e59-c42b-4a38-a6c8-85f903e73922/volumes"
Oct 30 19:44:54.396917 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.396895    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d898cf4d-a645-494a-a4af-88830acf24ab" path="/var/lib/kubelet/pods/d898cf4d-a645-494a-a4af-88830acf24ab/volumes"
Oct 30 19:44:54.810504 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.810470    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn"
Oct 30 19:44:54.978005 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.977902    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7" event={"ID":"eec1b428-31dc-4bb4-812e-c575b780ea7f","Type":"ContainerStarted","Data":"04ca98c19f1acd4e62bdba8361369ea1cb761265f48b3d0c63baa92cba4055c6"}
Oct 30 19:44:54.978005 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.977965    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7" event={"ID":"eec1b428-31dc-4bb4-812e-c575b780ea7f","Type":"ContainerStarted","Data":"1f6debeb71cf833889d0d10a10954c9cee8b46a4e76dfe22f28a9fc0457f35ce"}
Oct 30 19:44:54.978422 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.978405    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"
Oct 30 19:44:54.984279 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.983834    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842" event={"ID":"0e38d835-2f71-40a8-932e-1b6328c9f650","Type":"ContainerStarted","Data":"a6449277f04a7994ad5176d01956e51e8e005f5174d59d7cdb3c80a3bdc289ae"}
Oct 30 19:44:54.984279 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:54.983894    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842" event={"ID":"0e38d835-2f71-40a8-932e-1b6328c9f650","Type":"ContainerStarted","Data":"5661d4af344ba83a1a16c5d069d0c86aae4887ab086dd1db03d4996668b9451f"}
Oct 30 19:44:55.001996 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:55.001927    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7" podStartSLOduration=194.586790895 podStartE2EDuration="3m15.001906818s" totalImagesPullingTime="415.115923ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:41:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:55.00075526 +0000 UTC m=+2326.741746363" watchObservedRunningTime="2026-10-30 19:44:55.001906818 +0000 UTC m=+2326.742897899"
Oct 30 19:44:55.041570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:55.041512    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842" podStartSLOduration=187.041496818 podStartE2EDuration="3m7.041496818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:41:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:44:55.037419826 +0000 UTC m=+2326.778410906" watchObservedRunningTime="2026-10-30 19:44:55.041496818 +0000 UTC m=+2326.782487896"
Oct 30 19:44:55.826719 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:55.826684    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"]
Oct 30 19:44:55.827148 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:55.826905    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd" podUID="39af7415-b31d-4bc0-8a67-ec591643f5c8" containerName="istio-proxy" containerID="cri-o://36aac923525c3f0c5b28d22b373ecddb1ab45d7994b1114681459b78914b4dd8" gracePeriod=30
Oct 30 19:44:55.919980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:55.919943    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:55.925937 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:55.925904    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:55.995398 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:55.995364    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:44:55.996589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:44:55.996563    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:45:00.169206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.168827    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq"]
Oct 30 19:45:00.199194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.188484    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7"]
Oct 30 19:45:00.199194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.189071    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq"
Oct 30 19:45:00.209433 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.209408    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7"
Oct 30 19:45:00.215720 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.215694    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq"]
Oct 30 19:45:00.223565 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.223535    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7"]
Oct 30 19:45:00.255790 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.255756    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-z26xs\""
Oct 30 19:45:00.295371 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.295322    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0214fa4f-78a5-412a-b581-68c8244e73e8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7\" (UID: \"0214fa4f-78a5-412a-b581-68c8244e73e8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7"
Oct 30 19:45:00.295665 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.295639    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/378763b7-0302-43cd-bb31-32f2605d3374-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq\" (UID: \"378763b7-0302-43cd-bb31-32f2605d3374\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq"
Oct 30 19:45:00.295774 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.295759    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzhcr\" (UniqueName: \"kubernetes.io/projected/0214fa4f-78a5-412a-b581-68c8244e73e8-kube-api-access-zzhcr\") pod \"maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7\" (UID: \"0214fa4f-78a5-412a-b581-68c8244e73e8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7"
Oct 30 19:45:00.295841 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.295786    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbkxx\" (UniqueName: \"kubernetes.io/projected/378763b7-0302-43cd-bb31-32f2605d3374-kube-api-access-fbkxx\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq\" (UID: \"378763b7-0302-43cd-bb31-32f2605d3374\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq"
Oct 30 19:45:00.397604 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.397573    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/378763b7-0302-43cd-bb31-32f2605d3374-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq\" (UID: \"378763b7-0302-43cd-bb31-32f2605d3374\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq"
Oct 30 19:45:00.397804 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.397638    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzhcr\" (UniqueName: \"kubernetes.io/projected/0214fa4f-78a5-412a-b581-68c8244e73e8-kube-api-access-zzhcr\") pod \"maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7\" (UID: \"0214fa4f-78a5-412a-b581-68c8244e73e8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7"
Oct 30 19:45:00.397804 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.397666    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbkxx\" (UniqueName: \"kubernetes.io/projected/378763b7-0302-43cd-bb31-32f2605d3374-kube-api-access-fbkxx\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq\" (UID: \"378763b7-0302-43cd-bb31-32f2605d3374\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq"
Oct 30 19:45:00.397804 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.397778    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0214fa4f-78a5-412a-b581-68c8244e73e8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7\" (UID: \"0214fa4f-78a5-412a-b581-68c8244e73e8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7"
Oct 30 19:45:00.398371 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.398352    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/378763b7-0302-43cd-bb31-32f2605d3374-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq\" (UID: \"378763b7-0302-43cd-bb31-32f2605d3374\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq"
Oct 30 19:45:00.398454 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.398424    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0214fa4f-78a5-412a-b581-68c8244e73e8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7\" (UID: \"0214fa4f-78a5-412a-b581-68c8244e73e8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7"
Oct 30 19:45:00.409538 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.409507    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbkxx\" (UniqueName: \"kubernetes.io/projected/378763b7-0302-43cd-bb31-32f2605d3374-kube-api-access-fbkxx\") pod \"maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq\" (UID: \"378763b7-0302-43cd-bb31-32f2605d3374\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq"
Oct 30 19:45:00.409759 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.409732    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzhcr\" (UniqueName: \"kubernetes.io/projected/0214fa4f-78a5-412a-b581-68c8244e73e8-kube-api-access-zzhcr\") pod \"maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7\" (UID: \"0214fa4f-78a5-412a-b581-68c8244e73e8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7"
Oct 30 19:45:00.522249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.522212    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq"
Oct 30 19:45:00.534508 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.534483    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7"
Oct 30 19:45:00.829732 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.829701    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq"]
Oct 30 19:45:00.833140 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:45:00.833114    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod378763b7_0302_43cd_bb31_32f2605d3374.slice/crio-7db92f6aa36c49f9f346d426f4a3a096dd5677302079659eff4de5dde4f8ec65 WatchSource:0}: Error finding container 7db92f6aa36c49f9f346d426f4a3a096dd5677302079659eff4de5dde4f8ec65: Status 404 returned error can't find the container with id 7db92f6aa36c49f9f346d426f4a3a096dd5677302079659eff4de5dde4f8ec65
Oct 30 19:45:00.849942 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:00.849907    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7"]
Oct 30 19:45:00.852438 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:45:00.852401    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0214fa4f_78a5_412a_b581_68c8244e73e8.slice/crio-2568fc0eac886db87387d4b5e044376b7786ce25ac9f8be2200e6b6f6c928c1e WatchSource:0}: Error finding container 2568fc0eac886db87387d4b5e044376b7786ce25ac9f8be2200e6b6f6c928c1e: Status 404 returned error can't find the container with id 2568fc0eac886db87387d4b5e044376b7786ce25ac9f8be2200e6b6f6c928c1e
Oct 30 19:45:01.107145 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.107093    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq" event={"ID":"378763b7-0302-43cd-bb31-32f2605d3374","Type":"ContainerStarted","Data":"7db92f6aa36c49f9f346d426f4a3a096dd5677302079659eff4de5dde4f8ec65"}
Oct 30 19:45:01.109416 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.109386    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7" event={"ID":"0214fa4f-78a5-412a-b581-68c8244e73e8","Type":"ContainerStarted","Data":"2568fc0eac886db87387d4b5e044376b7786ce25ac9f8be2200e6b6f6c928c1e"}
Oct 30 19:45:01.111013 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.110990    4524 generic.go:356] "Generic (PLEG): container finished" podID="39af7415-b31d-4bc0-8a67-ec591643f5c8" containerID="36aac923525c3f0c5b28d22b373ecddb1ab45d7994b1114681459b78914b4dd8" exitCode=0
Oct 30 19:45:01.111144 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.111067    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd" event={"ID":"39af7415-b31d-4bc0-8a67-ec591643f5c8","Type":"ContainerDied","Data":"36aac923525c3f0c5b28d22b373ecddb1ab45d7994b1114681459b78914b4dd8"}
Oct 30 19:45:01.111981 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.111962    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:45:01.191909 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.191870    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"]
Oct 30 19:45:01.193249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.193202    4524 cpu_manager.go:414] "containerMap: removing container" podUID="39af7415-b31d-4bc0-8a67-ec591643f5c8" containerName="istio-proxy"
Oct 30 19:45:01.193249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.193235    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="39af7415-b31d-4bc0-8a67-ec591643f5c8" containerName="istio-proxy"
Oct 30 19:45:01.193478 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.193463    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="39af7415-b31d-4bc0-8a67-ec591643f5c8" containerName="istio-proxy"
Oct 30 19:45:01.203807 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.201274    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"]
Oct 30 19:45:01.203807 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.201385    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"
Oct 30 19:45:01.212935 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.211303    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-1a844e-serving-cert\""
Oct 30 19:45:01.212935 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.211866    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-workload-socket\") pod \"39af7415-b31d-4bc0-8a67-ec591643f5c8\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") "
Oct 30 19:45:01.212935 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.211910    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-credential-socket\") pod \"39af7415-b31d-4bc0-8a67-ec591643f5c8\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") "
Oct 30 19:45:01.212935 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.211948    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-podinfo\") pod \"39af7415-b31d-4bc0-8a67-ec591643f5c8\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") "
Oct 30 19:45:01.212935 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.211977    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/39af7415-b31d-4bc0-8a67-ec591643f5c8-kube-api-access-xr4pd\" (UniqueName: \"kubernetes.io/projected/39af7415-b31d-4bc0-8a67-ec591643f5c8-kube-api-access-xr4pd\") pod \"39af7415-b31d-4bc0-8a67-ec591643f5c8\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") "
Oct 30 19:45:01.212935 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.212054    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-data\") pod \"39af7415-b31d-4bc0-8a67-ec591643f5c8\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") "
Oct 30 19:45:01.212935 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.212212    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-workload-certs\") pod \"39af7415-b31d-4bc0-8a67-ec591643f5c8\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") "
Oct 30 19:45:01.212935 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.212286    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/39af7415-b31d-4bc0-8a67-ec591643f5c8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39af7415-b31d-4bc0-8a67-ec591643f5c8-istiod-ca-cert\") pod \"39af7415-b31d-4bc0-8a67-ec591643f5c8\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") "
Oct 30 19:45:01.215375 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.215112    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39af7415-b31d-4bc0-8a67-ec591643f5c8-istiod-ca-cert" pod "39af7415-b31d-4bc0-8a67-ec591643f5c8" (UID: "39af7415-b31d-4bc0-8a67-ec591643f5c8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:01.218505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.218445    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-data" pod "39af7415-b31d-4bc0-8a67-ec591643f5c8" (UID: "39af7415-b31d-4bc0-8a67-ec591643f5c8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:01.218678 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.218654    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-credential-socket" pod "39af7415-b31d-4bc0-8a67-ec591643f5c8" (UID: "39af7415-b31d-4bc0-8a67-ec591643f5c8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:01.218850 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.218830    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-workload-socket" pod "39af7415-b31d-4bc0-8a67-ec591643f5c8" (UID: "39af7415-b31d-4bc0-8a67-ec591643f5c8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:01.218928 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.218877    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-token\" (UniqueName: \"kubernetes.io/projected/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-token\") pod \"39af7415-b31d-4bc0-8a67-ec591643f5c8\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") "
Oct 30 19:45:01.219072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.219052    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-envoy\") pod \"39af7415-b31d-4bc0-8a67-ec591643f5c8\" (UID: \"39af7415-b31d-4bc0-8a67-ec591643f5c8\") "
Oct 30 19:45:01.219927 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.219902    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:01.220010 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.219932    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:01.220010 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.219947    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:01.220010 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.219962    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39af7415-b31d-4bc0-8a67-ec591643f5c8-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:01.223230 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.222388    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-podinfo" pod "39af7415-b31d-4bc0-8a67-ec591643f5c8" (UID: "39af7415-b31d-4bc0-8a67-ec591643f5c8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:45:01.224873 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.224844    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-workload-certs" pod "39af7415-b31d-4bc0-8a67-ec591643f5c8" (UID: "39af7415-b31d-4bc0-8a67-ec591643f5c8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:01.225373 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.225258    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-envoy" pod "39af7415-b31d-4bc0-8a67-ec591643f5c8" (UID: "39af7415-b31d-4bc0-8a67-ec591643f5c8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:01.264958 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.264913    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-token" pod "39af7415-b31d-4bc0-8a67-ec591643f5c8" (UID: "39af7415-b31d-4bc0-8a67-ec591643f5c8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:01.280500 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.280436    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39af7415-b31d-4bc0-8a67-ec591643f5c8-kube-api-access-xr4pd" pod "39af7415-b31d-4bc0-8a67-ec591643f5c8" (UID: "39af7415-b31d-4bc0-8a67-ec591643f5c8"). InnerVolumeSpecName "kube-api-access-xr4pd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:01.324860 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.324802    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08eef1b9-f940-48f4-bb6d-b355616ded55-metrics-certs\") pod \"maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d\" (UID: \"08eef1b9-f940-48f4-bb6d-b355616ded55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"
Oct 30 19:45:01.326682 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.326638    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzlqs\" (UniqueName: \"kubernetes.io/projected/08eef1b9-f940-48f4-bb6d-b355616ded55-kube-api-access-wzlqs\") pod \"maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d\" (UID: \"08eef1b9-f940-48f4-bb6d-b355616ded55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"
Oct 30 19:45:01.326914 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.326873    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08eef1b9-f940-48f4-bb6d-b355616ded55-maas-api-tls\") pod \"maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d\" (UID: \"08eef1b9-f940-48f4-bb6d-b355616ded55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"
Oct 30 19:45:01.327402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.327378    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08eef1b9-f940-48f4-bb6d-b355616ded55-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d\" (UID: \"08eef1b9-f940-48f4-bb6d-b355616ded55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"
Oct 30 19:45:01.327915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.327876    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:01.328044 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.328030    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:01.328119 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.328108    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39af7415-b31d-4bc0-8a67-ec591643f5c8-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:01.328205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.328190    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xr4pd\" (UniqueName: \"kubernetes.io/projected/39af7415-b31d-4bc0-8a67-ec591643f5c8-kube-api-access-xr4pd\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:01.328346 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.328330    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39af7415-b31d-4bc0-8a67-ec591643f5c8-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:01.430080 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.430034    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08eef1b9-f940-48f4-bb6d-b355616ded55-metrics-certs\") pod \"maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d\" (UID: \"08eef1b9-f940-48f4-bb6d-b355616ded55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"
Oct 30 19:45:01.430296 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.430116    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzlqs\" (UniqueName: \"kubernetes.io/projected/08eef1b9-f940-48f4-bb6d-b355616ded55-kube-api-access-wzlqs\") pod \"maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d\" (UID: \"08eef1b9-f940-48f4-bb6d-b355616ded55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"
Oct 30 19:45:01.430296 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.430154    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08eef1b9-f940-48f4-bb6d-b355616ded55-maas-api-tls\") pod \"maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d\" (UID: \"08eef1b9-f940-48f4-bb6d-b355616ded55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"
Oct 30 19:45:01.430432 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.430331    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08eef1b9-f940-48f4-bb6d-b355616ded55-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d\" (UID: \"08eef1b9-f940-48f4-bb6d-b355616ded55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"
Oct 30 19:45:01.431312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.431249    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08eef1b9-f940-48f4-bb6d-b355616ded55-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d\" (UID: \"08eef1b9-f940-48f4-bb6d-b355616ded55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"
Oct 30 19:45:01.433592 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.433564    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08eef1b9-f940-48f4-bb6d-b355616ded55-metrics-certs\") pod \"maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d\" (UID: \"08eef1b9-f940-48f4-bb6d-b355616ded55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"
Oct 30 19:45:01.434402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.434380    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08eef1b9-f940-48f4-bb6d-b355616ded55-maas-api-tls\") pod \"maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d\" (UID: \"08eef1b9-f940-48f4-bb6d-b355616ded55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"
Oct 30 19:45:01.442803 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.442780    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzlqs\" (UniqueName: \"kubernetes.io/projected/08eef1b9-f940-48f4-bb6d-b355616ded55-kube-api-access-wzlqs\") pod \"maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d\" (UID: \"08eef1b9-f940-48f4-bb6d-b355616ded55\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"
Oct 30 19:45:01.586238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.552810    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"
Oct 30 19:45:01.893383 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:01.893352    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"]
Oct 30 19:45:01.894918 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:45:01.894888    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08eef1b9_f940_48f4_bb6d_b355616ded55.slice/crio-2b4b362ef8e691306bf1d228c225fd586766297b5e20158622b53f3fcaed4744 WatchSource:0}: Error finding container 2b4b362ef8e691306bf1d228c225fd586766297b5e20158622b53f3fcaed4744: Status 404 returned error can't find the container with id 2b4b362ef8e691306bf1d228c225fd586766297b5e20158622b53f3fcaed4744
Oct 30 19:45:02.018124 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:02.018087    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"
Oct 30 19:45:02.124363 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:02.124319    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d" event={"ID":"08eef1b9-f940-48f4-bb6d-b355616ded55","Type":"ContainerStarted","Data":"2b4b362ef8e691306bf1d228c225fd586766297b5e20158622b53f3fcaed4744"}
Oct 30 19:45:02.126737 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:02.126701    4524 generic.go:356] "Generic (PLEG): container finished" podID="378763b7-0302-43cd-bb31-32f2605d3374" containerID="8f8e3030fca4b81f08f0b208a4ea2e22ead377dddda5571f892ba9aefce36ef3" exitCode=60
Oct 30 19:45:02.126912 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:02.126884    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq" event={"ID":"378763b7-0302-43cd-bb31-32f2605d3374","Type":"ContainerDied","Data":"8f8e3030fca4b81f08f0b208a4ea2e22ead377dddda5571f892ba9aefce36ef3"}
Oct 30 19:45:02.127481 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:02.127445    4524 scope.go:122] "RemoveContainer" containerID="8f8e3030fca4b81f08f0b208a4ea2e22ead377dddda5571f892ba9aefce36ef3"
Oct 30 19:45:02.133103 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:02.130926    4524 generic.go:356] "Generic (PLEG): container finished" podID="0214fa4f-78a5-412a-b581-68c8244e73e8" containerID="e82f12acc3d68907c1cba43fe3f726d9577dda76ce4254f4dd01ccd1257582b0" exitCode=60
Oct 30 19:45:02.133103 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:02.131048    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7" event={"ID":"0214fa4f-78a5-412a-b581-68c8244e73e8","Type":"ContainerDied","Data":"e82f12acc3d68907c1cba43fe3f726d9577dda76ce4254f4dd01ccd1257582b0"}
Oct 30 19:45:02.133103 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:02.131514    4524 scope.go:122] "RemoveContainer" containerID="e82f12acc3d68907c1cba43fe3f726d9577dda76ce4254f4dd01ccd1257582b0"
Oct 30 19:45:02.141862 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:02.135388    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd" event={"ID":"39af7415-b31d-4bc0-8a67-ec591643f5c8","Type":"ContainerDied","Data":"01b6df12aceeaaeab6063549c537626089388fb3646271774c0de32984452811"}
Oct 30 19:45:02.141862 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:02.135444    4524 scope.go:122] "RemoveContainer" containerID="36aac923525c3f0c5b28d22b373ecddb1ab45d7994b1114681459b78914b4dd8"
Oct 30 19:45:02.141862 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:02.135652    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"
Oct 30 19:45:02.215679 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:02.215290    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"]
Oct 30 19:45:02.219975 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:02.219912    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-lm7vd"]
Oct 30 19:45:02.396660 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:02.396612    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="39af7415-b31d-4bc0-8a67-ec591643f5c8" path="/var/lib/kubelet/pods/39af7415-b31d-4bc0-8a67-ec591643f5c8/volumes"
Oct 30 19:45:03.173661 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:03.173618    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq" event={"ID":"378763b7-0302-43cd-bb31-32f2605d3374","Type":"ContainerStarted","Data":"e04a3ce0dc4e15231dbbe7e38547e2e4d27d27a1725598571666fed7fe1a6a3d"}
Oct 30 19:45:03.179211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:03.179157    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7" event={"ID":"0214fa4f-78a5-412a-b581-68c8244e73e8","Type":"ContainerStarted","Data":"f83024447ccbff4625460fc7688abe6aabfb1974bfea1e280510b28c72c585e0"}
Oct 30 19:45:03.187696 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:03.187658    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d" event={"ID":"08eef1b9-f940-48f4-bb6d-b355616ded55","Type":"ContainerStarted","Data":"fc4b9846c54439357da91972c646fcfd66275ef88512b4c6dfc57456371da239"}
Oct 30 19:45:03.188635 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:03.188560    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"
Oct 30 19:45:03.216756 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:03.216689    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq" podStartSLOduration=3.2166679990000002 podStartE2EDuration="3.216667999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:03.211154859 +0000 UTC m=+2334.952145968" watchObservedRunningTime="2026-10-30 19:45:03.216667999 +0000 UTC m=+2334.957659079"
Oct 30 19:45:03.242274 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:03.242213    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d" podStartSLOduration=4.801507243 podStartE2EDuration="5.242194018s" totalImagesPullingTime="440.686775ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:03.234237936 +0000 UTC m=+2334.975229015" watchObservedRunningTime="2026-10-30 19:45:03.242194018 +0000 UTC m=+2334.983185094"
Oct 30 19:45:03.273112 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:03.273041    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7" podStartSLOduration=3.273003937 podStartE2EDuration="3.273003937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:03.264591288 +0000 UTC m=+2335.005582371" watchObservedRunningTime="2026-10-30 19:45:03.273003937 +0000 UTC m=+2335.013995017"
Oct 30 19:45:04.238664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:04.238623    4524 generic.go:356] "Generic (PLEG): container finished" podID="378763b7-0302-43cd-bb31-32f2605d3374" containerID="e04a3ce0dc4e15231dbbe7e38547e2e4d27d27a1725598571666fed7fe1a6a3d" exitCode=60
Oct 30 19:45:04.239038 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:04.238807    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq" event={"ID":"378763b7-0302-43cd-bb31-32f2605d3374","Type":"ContainerDied","Data":"e04a3ce0dc4e15231dbbe7e38547e2e4d27d27a1725598571666fed7fe1a6a3d"}
Oct 30 19:45:04.239038 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:04.238907    4524 scope.go:122] "RemoveContainer" containerID="8f8e3030fca4b81f08f0b208a4ea2e22ead377dddda5571f892ba9aefce36ef3"
Oct 30 19:45:04.240772 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:04.240748    4524 scope.go:122] "RemoveContainer" containerID="e04a3ce0dc4e15231dbbe7e38547e2e4d27d27a1725598571666fed7fe1a6a3d"
Oct 30 19:45:04.241504 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:45:04.241352    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq_odh-ai-gateway-infra(378763b7-0302-43cd-bb31-32f2605d3374)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq" podUID="378763b7-0302-43cd-bb31-32f2605d3374"
Oct 30 19:45:04.243707 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:04.243675    4524 generic.go:356] "Generic (PLEG): container finished" podID="0214fa4f-78a5-412a-b581-68c8244e73e8" containerID="f83024447ccbff4625460fc7688abe6aabfb1974bfea1e280510b28c72c585e0" exitCode=60
Oct 30 19:45:04.244104 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:04.244085    4524 scope.go:122] "RemoveContainer" containerID="f83024447ccbff4625460fc7688abe6aabfb1974bfea1e280510b28c72c585e0"
Oct 30 19:45:04.244479 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:45:04.244457    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7_odh-ai-gateway-infra(0214fa4f-78a5-412a-b581-68c8244e73e8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7" podUID="0214fa4f-78a5-412a-b581-68c8244e73e8"
Oct 30 19:45:04.244616 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:04.244527    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7" event={"ID":"0214fa4f-78a5-412a-b581-68c8244e73e8","Type":"ContainerDied","Data":"f83024447ccbff4625460fc7688abe6aabfb1974bfea1e280510b28c72c585e0"}
Oct 30 19:45:04.269700 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:04.269673    4524 scope.go:122] "RemoveContainer" containerID="e82f12acc3d68907c1cba43fe3f726d9577dda76ce4254f4dd01ccd1257582b0"
Oct 30 19:45:05.285194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:05.280491    4524 scope.go:122] "RemoveContainer" containerID="e04a3ce0dc4e15231dbbe7e38547e2e4d27d27a1725598571666fed7fe1a6a3d"
Oct 30 19:45:05.285194 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:45:05.280941    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq_odh-ai-gateway-infra(378763b7-0302-43cd-bb31-32f2605d3374)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq" podUID="378763b7-0302-43cd-bb31-32f2605d3374"
Oct 30 19:45:05.302553 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:05.302468    4524 scope.go:122] "RemoveContainer" containerID="f83024447ccbff4625460fc7688abe6aabfb1974bfea1e280510b28c72c585e0"
Oct 30 19:45:05.309529 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:45:05.302897    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7_odh-ai-gateway-infra(0214fa4f-78a5-412a-b581-68c8244e73e8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7" podUID="0214fa4f-78a5-412a-b581-68c8244e73e8"
Oct 30 19:45:06.367527 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:06.367493    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj"]
Oct 30 19:45:09.251559 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:09.251530    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"
Oct 30 19:45:11.368962 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:11.368895    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj" podUID="ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4" containerName="payload-processing" containerID="cri-o://aba3b344a54a8264039bc599981f0d9f00a5bae3849a2d1f632917839811a8a6" gracePeriod=25
Oct 30 19:45:11.799662 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:11.799625    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj"
Oct 30 19:45:11.841962 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:11.841914    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh"]
Oct 30 19:45:11.843149 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:11.843122    4524 cpu_manager.go:414] "containerMap: removing container" podUID="ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4" containerName="payload-processing"
Oct 30 19:45:11.843149 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:11.843145    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4" containerName="payload-processing"
Oct 30 19:45:11.843409 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:11.843365    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4" containerName="payload-processing"
Oct 30 19:45:11.847544 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:11.847522    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh"
Oct 30 19:45:11.855287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:11.855132    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-826518\""
Oct 30 19:45:11.862067 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:11.862042    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh"]
Oct 30 19:45:11.881708 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:11.881677    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-826518-dockercfg-g6hrn\""
Oct 30 19:45:11.944061 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:11.943966    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4-kube-api-access-wf77p\" (UniqueName: \"kubernetes.io/projected/ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4-kube-api-access-wf77p\") pod \"ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4\" (UID: \"ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4\") "
Oct 30 19:45:11.944264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:11.944123    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4-plugins-config-volume\") pod \"ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4\" (UID: \"ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4\") "
Oct 30 19:45:11.944788 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:11.944763    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4-plugins-config-volume" pod "ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4" (UID: "ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:11.945214 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:11.945153    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/776185e2-f0f0-4a7e-b4ee-3d5f2e429f46-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh\" (UID: \"776185e2-f0f0-4a7e-b4ee-3d5f2e429f46\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh"
Oct 30 19:45:11.945428 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:11.945402    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbn4t\" (UniqueName: \"kubernetes.io/projected/776185e2-f0f0-4a7e-b4ee-3d5f2e429f46-kube-api-access-gbn4t\") pod \"payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh\" (UID: \"776185e2-f0f0-4a7e-b4ee-3d5f2e429f46\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh"
Oct 30 19:45:11.945718 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:11.945690    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:11.947715 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:11.947611    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4-kube-api-access-wf77p" pod "ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4" (UID: "ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4"). InnerVolumeSpecName "kube-api-access-wf77p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:12.048263 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:12.048201    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbn4t\" (UniqueName: \"kubernetes.io/projected/776185e2-f0f0-4a7e-b4ee-3d5f2e429f46-kube-api-access-gbn4t\") pod \"payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh\" (UID: \"776185e2-f0f0-4a7e-b4ee-3d5f2e429f46\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh"
Oct 30 19:45:12.048573 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:12.048534    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/776185e2-f0f0-4a7e-b4ee-3d5f2e429f46-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh\" (UID: \"776185e2-f0f0-4a7e-b4ee-3d5f2e429f46\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh"
Oct 30 19:45:12.053254 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:12.048792    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wf77p\" (UniqueName: \"kubernetes.io/projected/ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4-kube-api-access-wf77p\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:12.053254 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:12.049761    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/776185e2-f0f0-4a7e-b4ee-3d5f2e429f46-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh\" (UID: \"776185e2-f0f0-4a7e-b4ee-3d5f2e429f46\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh"
Oct 30 19:45:12.075216 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:12.074005    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbn4t\" (UniqueName: \"kubernetes.io/projected/776185e2-f0f0-4a7e-b4ee-3d5f2e429f46-kube-api-access-gbn4t\") pod \"payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh\" (UID: \"776185e2-f0f0-4a7e-b4ee-3d5f2e429f46\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh"
Oct 30 19:45:12.172192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:12.171471    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh"
Oct 30 19:45:12.422736 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:12.421699    4524 generic.go:356] "Generic (PLEG): container finished" podID="ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4" containerID="aba3b344a54a8264039bc599981f0d9f00a5bae3849a2d1f632917839811a8a6" exitCode=0
Oct 30 19:45:12.422736 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:12.421808    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj" event={"ID":"ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4","Type":"ContainerDied","Data":"aba3b344a54a8264039bc599981f0d9f00a5bae3849a2d1f632917839811a8a6"}
Oct 30 19:45:12.422736 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:12.421858    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj" event={"ID":"ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4","Type":"ContainerDied","Data":"6a5869be1dfe1fadca2b31389a4e20ac89bbde69c53670063c272693ba9b6d36"}
Oct 30 19:45:12.422736 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:12.421887    4524 scope.go:122] "RemoveContainer" containerID="aba3b344a54a8264039bc599981f0d9f00a5bae3849a2d1f632917839811a8a6"
Oct 30 19:45:12.422736 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:12.422116    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj"
Oct 30 19:45:12.447402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:12.447370    4524 scope.go:122] "RemoveContainer" containerID="aba3b344a54a8264039bc599981f0d9f00a5bae3849a2d1f632917839811a8a6"
Oct 30 19:45:12.447995 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:12.447975    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj"]
Oct 30 19:45:12.448537 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:45:12.448456    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aba3b344a54a8264039bc599981f0d9f00a5bae3849a2d1f632917839811a8a6\": container with ID starting with aba3b344a54a8264039bc599981f0d9f00a5bae3849a2d1f632917839811a8a6 not found: ID does not exist" containerID="aba3b344a54a8264039bc599981f0d9f00a5bae3849a2d1f632917839811a8a6"
Oct 30 19:45:12.448647 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:12.448573    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aba3b344a54a8264039bc599981f0d9f00a5bae3849a2d1f632917839811a8a6"} err="failed to get container status \"aba3b344a54a8264039bc599981f0d9f00a5bae3849a2d1f632917839811a8a6\": rpc error: code = NotFound desc = could not find container \"aba3b344a54a8264039bc599981f0d9f00a5bae3849a2d1f632917839811a8a6\": container with ID starting with aba3b344a54a8264039bc599981f0d9f00a5bae3849a2d1f632917839811a8a6 not found: ID does not exist"
Oct 30 19:45:12.455579 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:12.455552    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7aa217-7fdc6f87b6-bbnsj"]
Oct 30 19:45:12.652674 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:12.652640    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh"]
Oct 30 19:45:12.652964 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:45:12.652941    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod776185e2_f0f0_4a7e_b4ee_3d5f2e429f46.slice/crio-c35afac143b070c9b4878acdbb6fa5d75429752ce5b6116bf96d649c5cae48b9 WatchSource:0}: Error finding container c35afac143b070c9b4878acdbb6fa5d75429752ce5b6116bf96d649c5cae48b9: Status 404 returned error can't find the container with id c35afac143b070c9b4878acdbb6fa5d75429752ce5b6116bf96d649c5cae48b9
Oct 30 19:45:13.282196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:13.279000    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"]
Oct 30 19:45:13.282196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:13.279225    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842" podUID="0e38d835-2f71-40a8-932e-1b6328c9f650" containerName="istio-proxy" containerID="cri-o://a6449277f04a7994ad5176d01956e51e8e005f5174d59d7cdb3c80a3bdc289ae" gracePeriod=30
Oct 30 19:45:13.493198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:13.490695    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh" event={"ID":"776185e2-f0f0-4a7e-b4ee-3d5f2e429f46","Type":"ContainerStarted","Data":"52427cdffc62476c567ede3a700a3c34a3e8291c6ff22bb0e4f213e51d2e520a"}
Oct 30 19:45:13.493198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:13.490762    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh" event={"ID":"776185e2-f0f0-4a7e-b4ee-3d5f2e429f46","Type":"ContainerStarted","Data":"c35afac143b070c9b4878acdbb6fa5d75429752ce5b6116bf96d649c5cae48b9"}
Oct 30 19:45:13.493198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:13.491895    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh"
Oct 30 19:45:13.565646 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:13.560616    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh" podStartSLOduration=3.560598593 podStartE2EDuration="3.560598593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:13.559904454 +0000 UTC m=+2345.300895534" watchObservedRunningTime="2026-10-30 19:45:13.560598593 +0000 UTC m=+2345.301589674"
Oct 30 19:45:14.416798 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:14.416737    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4" path="/var/lib/kubelet/pods/ca1c5222-0cc6-4cf7-8cc2-18855d8d52c4/volumes"
Oct 30 19:45:15.651078 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:45:15.651037    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:45:17.382909 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:17.382867    4524 scope.go:122] "RemoveContainer" containerID="f83024447ccbff4625460fc7688abe6aabfb1974bfea1e280510b28c72c585e0"
Oct 30 19:45:17.383452 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:17.383311    4524 scope.go:122] "RemoveContainer" containerID="e04a3ce0dc4e15231dbbe7e38547e2e4d27d27a1725598571666fed7fe1a6a3d"
Oct 30 19:45:18.628163 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.628135    4524 generic.go:356] "Generic (PLEG): container finished" podID="0214fa4f-78a5-412a-b581-68c8244e73e8" containerID="6a6e1bd977080fd7eeaf89d14a99e9e8a563d99c8a035bd0a17e680c97fe8778" exitCode=60
Oct 30 19:45:18.628517 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.628210    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7" event={"ID":"0214fa4f-78a5-412a-b581-68c8244e73e8","Type":"ContainerDied","Data":"6a6e1bd977080fd7eeaf89d14a99e9e8a563d99c8a035bd0a17e680c97fe8778"}
Oct 30 19:45:18.628517 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.628278    4524 scope.go:122] "RemoveContainer" containerID="f83024447ccbff4625460fc7688abe6aabfb1974bfea1e280510b28c72c585e0"
Oct 30 19:45:18.628796 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.628778    4524 scope.go:122] "RemoveContainer" containerID="6a6e1bd977080fd7eeaf89d14a99e9e8a563d99c8a035bd0a17e680c97fe8778"
Oct 30 19:45:18.629240 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:45:18.629208    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7_odh-ai-gateway-infra(0214fa4f-78a5-412a-b581-68c8244e73e8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7" podUID="0214fa4f-78a5-412a-b581-68c8244e73e8"
Oct 30 19:45:18.629863 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.629844    4524 generic.go:356] "Generic (PLEG): container finished" podID="0e38d835-2f71-40a8-932e-1b6328c9f650" containerID="a6449277f04a7994ad5176d01956e51e8e005f5174d59d7cdb3c80a3bdc289ae" exitCode=0
Oct 30 19:45:18.630135 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.630117    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842" event={"ID":"0e38d835-2f71-40a8-932e-1b6328c9f650","Type":"ContainerDied","Data":"a6449277f04a7994ad5176d01956e51e8e005f5174d59d7cdb3c80a3bdc289ae"}
Oct 30 19:45:18.630206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.630183    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842" event={"ID":"0e38d835-2f71-40a8-932e-1b6328c9f650","Type":"ContainerDied","Data":"5661d4af344ba83a1a16c5d069d0c86aae4887ab086dd1db03d4996668b9451f"}
Oct 30 19:45:18.630252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.630213    4524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5661d4af344ba83a1a16c5d069d0c86aae4887ab086dd1db03d4996668b9451f"
Oct 30 19:45:18.632337 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.632314    4524 generic.go:356] "Generic (PLEG): container finished" podID="378763b7-0302-43cd-bb31-32f2605d3374" containerID="b569d531a4abfe12a00422aa2669b121df072a1f69cdae6067ed344c7647a061" exitCode=60
Oct 30 19:45:18.632429 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.632386    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq" event={"ID":"378763b7-0302-43cd-bb31-32f2605d3374","Type":"ContainerDied","Data":"b569d531a4abfe12a00422aa2669b121df072a1f69cdae6067ed344c7647a061"}
Oct 30 19:45:18.632714 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.632692    4524 scope.go:122] "RemoveContainer" containerID="b569d531a4abfe12a00422aa2669b121df072a1f69cdae6067ed344c7647a061"
Oct 30 19:45:18.632963 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:45:18.632945    4524 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq_odh-ai-gateway-infra(378763b7-0302-43cd-bb31-32f2605d3374)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq" podUID="378763b7-0302-43cd-bb31-32f2605d3374"
Oct 30 19:45:18.727125 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.727075    4524 scope.go:122] "RemoveContainer" containerID="e04a3ce0dc4e15231dbbe7e38547e2e4d27d27a1725598571666fed7fe1a6a3d"
Oct 30 19:45:18.733250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.733226    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:45:18.834415 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.834380    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-token\" (UniqueName: \"kubernetes.io/projected/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-token\") pod \"0e38d835-2f71-40a8-932e-1b6328c9f650\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") "
Oct 30 19:45:18.834564 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.834459    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-podinfo\") pod \"0e38d835-2f71-40a8-932e-1b6328c9f650\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") "
Oct 30 19:45:18.834564 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.834513    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e38d835-2f71-40a8-932e-1b6328c9f650-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e38d835-2f71-40a8-932e-1b6328c9f650-istiod-ca-cert\") pod \"0e38d835-2f71-40a8-932e-1b6328c9f650\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") "
Oct 30 19:45:18.834662 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.834585    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-credential-socket\") pod \"0e38d835-2f71-40a8-932e-1b6328c9f650\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") "
Oct 30 19:45:18.834662 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.834639    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-data\") pod \"0e38d835-2f71-40a8-932e-1b6328c9f650\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") "
Oct 30 19:45:18.834766 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.834684    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-envoy\") pod \"0e38d835-2f71-40a8-932e-1b6328c9f650\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") "
Oct 30 19:45:18.834766 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.834715    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e38d835-2f71-40a8-932e-1b6328c9f650-kube-api-access-r27lb\" (UniqueName: \"kubernetes.io/projected/0e38d835-2f71-40a8-932e-1b6328c9f650-kube-api-access-r27lb\") pod \"0e38d835-2f71-40a8-932e-1b6328c9f650\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") "
Oct 30 19:45:18.834874 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.834788    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-workload-certs\") pod \"0e38d835-2f71-40a8-932e-1b6328c9f650\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") "
Oct 30 19:45:18.834948 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.834915    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-credential-socket" pod "0e38d835-2f71-40a8-932e-1b6328c9f650" (UID: "0e38d835-2f71-40a8-932e-1b6328c9f650"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:18.835001 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.834940    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e38d835-2f71-40a8-932e-1b6328c9f650-istiod-ca-cert" pod "0e38d835-2f71-40a8-932e-1b6328c9f650" (UID: "0e38d835-2f71-40a8-932e-1b6328c9f650"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:18.835084 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.835064    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-workload-socket\") pod \"0e38d835-2f71-40a8-932e-1b6328c9f650\" (UID: \"0e38d835-2f71-40a8-932e-1b6328c9f650\") "
Oct 30 19:45:18.835246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.835217    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-data" pod "0e38d835-2f71-40a8-932e-1b6328c9f650" (UID: "0e38d835-2f71-40a8-932e-1b6328c9f650"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:18.835342 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.835231    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-workload-certs" pod "0e38d835-2f71-40a8-932e-1b6328c9f650" (UID: "0e38d835-2f71-40a8-932e-1b6328c9f650"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:18.835506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.835487    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-workload-socket" pod "0e38d835-2f71-40a8-932e-1b6328c9f650" (UID: "0e38d835-2f71-40a8-932e-1b6328c9f650"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:18.836039 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.836017    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:18.836111 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.836047    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e38d835-2f71-40a8-932e-1b6328c9f650-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:18.836111 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.836062    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:18.836111 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.836077    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:18.836111 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.836091    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:18.837462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.837442    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-token" pod "0e38d835-2f71-40a8-932e-1b6328c9f650" (UID: "0e38d835-2f71-40a8-932e-1b6328c9f650"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:18.837527 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.837483    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e38d835-2f71-40a8-932e-1b6328c9f650-kube-api-access-r27lb" pod "0e38d835-2f71-40a8-932e-1b6328c9f650" (UID: "0e38d835-2f71-40a8-932e-1b6328c9f650"). InnerVolumeSpecName "kube-api-access-r27lb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:18.837527 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.837502    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-envoy" pod "0e38d835-2f71-40a8-932e-1b6328c9f650" (UID: "0e38d835-2f71-40a8-932e-1b6328c9f650"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:18.837671 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.837652    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-podinfo" pod "0e38d835-2f71-40a8-932e-1b6328c9f650" (UID: "0e38d835-2f71-40a8-932e-1b6328c9f650"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:45:18.938278 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.938243    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:18.938278 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.938277    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:18.938486 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.938287    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e38d835-2f71-40a8-932e-1b6328c9f650-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:18.938486 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:18.938301    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r27lb\" (UniqueName: \"kubernetes.io/projected/0e38d835-2f71-40a8-932e-1b6328c9f650-kube-api-access-r27lb\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:19.645728 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:19.645074    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"
Oct 30 19:45:19.658576 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:19.658551    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7"]
Oct 30 19:45:19.676239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:19.676166    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"]
Oct 30 19:45:19.681303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:19.681272    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq"]
Oct 30 19:45:19.686718 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:19.686620    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-6d9f17-openshift-default-bfb5c8b7f-ml842"]
Oct 30 19:45:19.960620 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:19.960593    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7"
Oct 30 19:45:19.994418 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:19.994392    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq"
Oct 30 19:45:20.098776 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.098749    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0214fa4f-78a5-412a-b581-68c8244e73e8-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0214fa4f-78a5-412a-b581-68c8244e73e8-maas-api-service-ca\") pod \"0214fa4f-78a5-412a-b581-68c8244e73e8\" (UID: \"0214fa4f-78a5-412a-b581-68c8244e73e8\") "
Oct 30 19:45:20.098939 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.098790    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/378763b7-0302-43cd-bb31-32f2605d3374-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/378763b7-0302-43cd-bb31-32f2605d3374-maas-api-service-ca\") pod \"378763b7-0302-43cd-bb31-32f2605d3374\" (UID: \"378763b7-0302-43cd-bb31-32f2605d3374\") "
Oct 30 19:45:20.099020 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.098968    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0214fa4f-78a5-412a-b581-68c8244e73e8-kube-api-access-zzhcr\" (UniqueName: \"kubernetes.io/projected/0214fa4f-78a5-412a-b581-68c8244e73e8-kube-api-access-zzhcr\") pod \"0214fa4f-78a5-412a-b581-68c8244e73e8\" (UID: \"0214fa4f-78a5-412a-b581-68c8244e73e8\") "
Oct 30 19:45:20.099020 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.099007    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/378763b7-0302-43cd-bb31-32f2605d3374-kube-api-access-fbkxx\" (UniqueName: \"kubernetes.io/projected/378763b7-0302-43cd-bb31-32f2605d3374-kube-api-access-fbkxx\") pod \"378763b7-0302-43cd-bb31-32f2605d3374\" (UID: \"378763b7-0302-43cd-bb31-32f2605d3374\") "
Oct 30 19:45:20.099225 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.099190    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0214fa4f-78a5-412a-b581-68c8244e73e8-maas-api-service-ca" pod "0214fa4f-78a5-412a-b581-68c8244e73e8" (UID: "0214fa4f-78a5-412a-b581-68c8244e73e8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:20.099372 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.099225    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/378763b7-0302-43cd-bb31-32f2605d3374-maas-api-service-ca" pod "378763b7-0302-43cd-bb31-32f2605d3374" (UID: "378763b7-0302-43cd-bb31-32f2605d3374"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:20.099694 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.099672    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0214fa4f-78a5-412a-b581-68c8244e73e8-maas-api-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:20.099694 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.099694    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/378763b7-0302-43cd-bb31-32f2605d3374-maas-api-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:20.101211 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.101183    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0214fa4f-78a5-412a-b581-68c8244e73e8-kube-api-access-zzhcr" pod "0214fa4f-78a5-412a-b581-68c8244e73e8" (UID: "0214fa4f-78a5-412a-b581-68c8244e73e8"). InnerVolumeSpecName "kube-api-access-zzhcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:20.101287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.101256    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/378763b7-0302-43cd-bb31-32f2605d3374-kube-api-access-fbkxx" pod "378763b7-0302-43cd-bb31-32f2605d3374" (UID: "378763b7-0302-43cd-bb31-32f2605d3374"). InnerVolumeSpecName "kube-api-access-fbkxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:20.201405 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.201339    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzhcr\" (UniqueName: \"kubernetes.io/projected/0214fa4f-78a5-412a-b581-68c8244e73e8-kube-api-access-zzhcr\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:20.201405 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.201366    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbkxx\" (UniqueName: \"kubernetes.io/projected/378763b7-0302-43cd-bb31-32f2605d3374-kube-api-access-fbkxx\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:20.463245 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.463099    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e38d835-2f71-40a8-932e-1b6328c9f650" path="/var/lib/kubelet/pods/0e38d835-2f71-40a8-932e-1b6328c9f650/volumes"
Oct 30 19:45:20.657955 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.657297    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq"
Oct 30 19:45:20.657955 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.657782    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq" event={"ID":"378763b7-0302-43cd-bb31-32f2605d3374","Type":"ContainerDied","Data":"7db92f6aa36c49f9f346d426f4a3a096dd5677302079659eff4de5dde4f8ec65"}
Oct 30 19:45:20.657955 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.657835    4524 scope.go:122] "RemoveContainer" containerID="b569d531a4abfe12a00422aa2669b121df072a1f69cdae6067ed344c7647a061"
Oct 30 19:45:20.660045 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.660016    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7" event={"ID":"0214fa4f-78a5-412a-b581-68c8244e73e8","Type":"ContainerDied","Data":"2568fc0eac886db87387d4b5e044376b7786ce25ac9f8be2200e6b6f6c928c1e"}
Oct 30 19:45:20.660159 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.660112    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7"
Oct 30 19:45:20.672254 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.672231    4524 scope.go:122] "RemoveContainer" containerID="6a6e1bd977080fd7eeaf89d14a99e9e8a563d99c8a035bd0a17e680c97fe8778"
Oct 30 19:45:20.682846 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.682822    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq"]
Oct 30 19:45:20.735248 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.734935    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-b281d5-29889825-scgmq"]
Oct 30 19:45:20.785194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.783680    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7"]
Oct 30 19:45:20.790734 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:20.790696    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-27b5d5-29889825-vl6t7"]
Oct 30 19:45:22.394413 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:22.394375    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0214fa4f-78a5-412a-b581-68c8244e73e8" path="/var/lib/kubelet/pods/0214fa4f-78a5-412a-b581-68c8244e73e8/volumes"
Oct 30 19:45:22.394936 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:22.394920    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="378763b7-0302-43cd-bb31-32f2605d3374" path="/var/lib/kubelet/pods/378763b7-0302-43cd-bb31-32f2605d3374/volumes"
Oct 30 19:45:24.166194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:24.150253    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d"]
Oct 30 19:45:24.187191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:24.182203    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g"]
Oct 30 19:45:24.839879 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:24.839847    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn"]
Oct 30 19:45:25.604872 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:25.604843    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh"
Oct 30 19:45:27.128585 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.128546    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"]
Oct 30 19:45:27.129425 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.129357    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql" podUID="1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd" containerName="maas-api" containerID="cri-o://7f8b518634a403388e1659c3b800eec20b4f544425c988518b164c095a851a4b" gracePeriod=30
Oct 30 19:45:27.738643 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.738608    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"]
Oct 30 19:45:27.740855 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.740145    4524 cpu_manager.go:414] "containerMap: removing container" podUID="378763b7-0302-43cd-bb31-32f2605d3374" containerName="cleanup"
Oct 30 19:45:27.741057 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.741035    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="378763b7-0302-43cd-bb31-32f2605d3374" containerName="cleanup"
Oct 30 19:45:27.741195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.741162    4524 cpu_manager.go:414] "containerMap: removing container" podUID="378763b7-0302-43cd-bb31-32f2605d3374" containerName="cleanup"
Oct 30 19:45:27.741306 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.741293    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="378763b7-0302-43cd-bb31-32f2605d3374" containerName="cleanup"
Oct 30 19:45:27.741422 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.741410    4524 cpu_manager.go:414] "containerMap: removing container" podUID="0214fa4f-78a5-412a-b581-68c8244e73e8" containerName="cleanup"
Oct 30 19:45:27.741515 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.741503    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0214fa4f-78a5-412a-b581-68c8244e73e8" containerName="cleanup"
Oct 30 19:45:27.741628 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.741617    4524 cpu_manager.go:414] "containerMap: removing container" podUID="0214fa4f-78a5-412a-b581-68c8244e73e8" containerName="cleanup"
Oct 30 19:45:27.741723 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.741711    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0214fa4f-78a5-412a-b581-68c8244e73e8" containerName="cleanup"
Oct 30 19:45:27.741816 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.741805    4524 cpu_manager.go:414] "containerMap: removing container" podUID="0214fa4f-78a5-412a-b581-68c8244e73e8" containerName="cleanup"
Oct 30 19:45:27.741902 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.741887    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0214fa4f-78a5-412a-b581-68c8244e73e8" containerName="cleanup"
Oct 30 19:45:27.742014 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.742003    4524 cpu_manager.go:414] "containerMap: removing container" podUID="0e38d835-2f71-40a8-932e-1b6328c9f650" containerName="istio-proxy"
Oct 30 19:45:27.742092 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.742081    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e38d835-2f71-40a8-932e-1b6328c9f650" containerName="istio-proxy"
Oct 30 19:45:27.742411 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.742393    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e38d835-2f71-40a8-932e-1b6328c9f650" containerName="istio-proxy"
Oct 30 19:45:27.742536 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.742525    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="378763b7-0302-43cd-bb31-32f2605d3374" containerName="cleanup"
Oct 30 19:45:27.742675 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.742659    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="378763b7-0302-43cd-bb31-32f2605d3374" containerName="cleanup"
Oct 30 19:45:27.742781 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.742771    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="0214fa4f-78a5-412a-b581-68c8244e73e8" containerName="cleanup"
Oct 30 19:45:27.742875 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.742865    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="0214fa4f-78a5-412a-b581-68c8244e73e8" containerName="cleanup"
Oct 30 19:45:27.742964 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.742954    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="0214fa4f-78a5-412a-b581-68c8244e73e8" containerName="cleanup"
Oct 30 19:45:27.749132 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.748620    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"]
Oct 30 19:45:27.749132 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.748735    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"
Oct 30 19:45:27.752971 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.752790    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-826518-serving-cert\""
Oct 30 19:45:27.942949 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.942750    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/662686fd-7d04-48b9-bf7e-adbd30299109-maas-api-tls\") pod \"maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj\" (UID: \"662686fd-7d04-48b9-bf7e-adbd30299109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"
Oct 30 19:45:27.942949 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.942851    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/662686fd-7d04-48b9-bf7e-adbd30299109-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj\" (UID: \"662686fd-7d04-48b9-bf7e-adbd30299109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"
Oct 30 19:45:27.943193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.943032    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/662686fd-7d04-48b9-bf7e-adbd30299109-metrics-certs\") pod \"maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj\" (UID: \"662686fd-7d04-48b9-bf7e-adbd30299109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"
Oct 30 19:45:27.943193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.943068    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5llwr\" (UniqueName: \"kubernetes.io/projected/662686fd-7d04-48b9-bf7e-adbd30299109-kube-api-access-5llwr\") pod \"maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj\" (UID: \"662686fd-7d04-48b9-bf7e-adbd30299109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"
Oct 30 19:45:27.950086 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.950050    4524 generic.go:356] "Generic (PLEG): container finished" podID="1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd" containerID="7f8b518634a403388e1659c3b800eec20b4f544425c988518b164c095a851a4b" exitCode=0
Oct 30 19:45:27.950343 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.950128    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql" event={"ID":"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd","Type":"ContainerDied","Data":"7f8b518634a403388e1659c3b800eec20b4f544425c988518b164c095a851a4b"}
Oct 30 19:45:27.950343 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.950248    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql" event={"ID":"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd","Type":"ContainerDied","Data":"c6a4ceb07164ecdd06595438b5924e29b43c2a21d09d0e4639fa7a95603c1974"}
Oct 30 19:45:27.950343 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.950291    4524 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c6a4ceb07164ecdd06595438b5924e29b43c2a21d09d0e4639fa7a95603c1974"
Oct 30 19:45:27.953896 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:27.953875    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"
Oct 30 19:45:28.044037 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.043997    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-kube-api-access-gwlnl\" (UniqueName: \"kubernetes.io/projected/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-kube-api-access-gwlnl\") pod \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\" (UID: \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\") "
Oct 30 19:45:28.044345 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.044067    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-maas-api-tls\") pod \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\" (UID: \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\") "
Oct 30 19:45:28.044345 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.044156    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-openshift-service-ca\") pod \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\" (UID: \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\") "
Oct 30 19:45:28.044345 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.044207    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-metrics-certs\") pod \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\" (UID: \"1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd\") "
Oct 30 19:45:28.044750 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.044725    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/662686fd-7d04-48b9-bf7e-adbd30299109-metrics-certs\") pod \"maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj\" (UID: \"662686fd-7d04-48b9-bf7e-adbd30299109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"
Oct 30 19:45:28.044823 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.044776    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5llwr\" (UniqueName: \"kubernetes.io/projected/662686fd-7d04-48b9-bf7e-adbd30299109-kube-api-access-5llwr\") pod \"maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj\" (UID: \"662686fd-7d04-48b9-bf7e-adbd30299109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"
Oct 30 19:45:28.044944 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.044927    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/662686fd-7d04-48b9-bf7e-adbd30299109-maas-api-tls\") pod \"maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj\" (UID: \"662686fd-7d04-48b9-bf7e-adbd30299109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"
Oct 30 19:45:28.045106 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.044995    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/662686fd-7d04-48b9-bf7e-adbd30299109-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj\" (UID: \"662686fd-7d04-48b9-bf7e-adbd30299109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"
Oct 30 19:45:28.045820 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:45:28.045459    4524 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-serving-cert: secret "maas-api-e2e-ipp-a-826518-serving-cert" not found
Oct 30 19:45:28.045820 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:45:28.045545    4524 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/662686fd-7d04-48b9-bf7e-adbd30299109-maas-api-tls podName:662686fd-7d04-48b9-bf7e-adbd30299109 nodeName:}" failed. No retries permitted until 2026-10-30 19:45:28.545522147 +0000 UTC m=+2360.286513209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/662686fd-7d04-48b9-bf7e-adbd30299109-maas-api-tls") pod "maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj" (UID: "662686fd-7d04-48b9-bf7e-adbd30299109") : secret "maas-api-e2e-ipp-a-826518-serving-cert" not found
Oct 30 19:45:28.045820 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.045750    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-openshift-service-ca" pod "1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd" (UID: "1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:28.046134 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.046112    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/662686fd-7d04-48b9-bf7e-adbd30299109-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj\" (UID: \"662686fd-7d04-48b9-bf7e-adbd30299109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"
Oct 30 19:45:28.050583 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.050558    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-metrics-certs" pod "1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd" (UID: "1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:28.050727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.050615    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-maas-api-tls" pod "1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd" (UID: "1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:28.052896 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.052868    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/662686fd-7d04-48b9-bf7e-adbd30299109-metrics-certs\") pod \"maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj\" (UID: \"662686fd-7d04-48b9-bf7e-adbd30299109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"
Oct 30 19:45:28.057919 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.057888    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5llwr\" (UniqueName: \"kubernetes.io/projected/662686fd-7d04-48b9-bf7e-adbd30299109-kube-api-access-5llwr\") pod \"maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj\" (UID: \"662686fd-7d04-48b9-bf7e-adbd30299109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"
Oct 30 19:45:28.058569 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.058545    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-kube-api-access-gwlnl" pod "1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd" (UID: "1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd"). InnerVolumeSpecName "kube-api-access-gwlnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:28.146676 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.146632    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwlnl\" (UniqueName: \"kubernetes.io/projected/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-kube-api-access-gwlnl\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:28.146676 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.146669    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:28.147116 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.146684    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:28.147116 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.146697    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:28.552242 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.552198    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/662686fd-7d04-48b9-bf7e-adbd30299109-maas-api-tls\") pod \"maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj\" (UID: \"662686fd-7d04-48b9-bf7e-adbd30299109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"
Oct 30 19:45:28.555161 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.555113    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/662686fd-7d04-48b9-bf7e-adbd30299109-maas-api-tls\") pod \"maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj\" (UID: \"662686fd-7d04-48b9-bf7e-adbd30299109\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"
Oct 30 19:45:28.695401 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.693382    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"
Oct 30 19:45:28.962892 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.962799    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"
Oct 30 19:45:28.995999 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:28.995963    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"]
Oct 30 19:45:29.003317 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:29.003285    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-618d93-64cf9b6bf-ncrql"]
Oct 30 19:45:29.048488 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:29.048446    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"]
Oct 30 19:45:29.151589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:29.151548    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d" podUID="97dda0db-86c4-4e0c-92af-55bb1e85e2f3" containerName="payload-processing" containerID="cri-o://f76675191dc14101d26001b1232f3c4a1660d1b752ab49f1fa3d55e4072332be" gracePeriod=25
Oct 30 19:45:29.183094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:29.183052    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g" podUID="b7b95d1a-dbee-45f7-806f-34a1b28a7fcf" containerName="payload-pre-processing" containerID="cri-o://f166492b87b0df6fa0e667b36463add7d7adb6a242a321d3da61230b1720a3e8" gracePeriod=25
Oct 30 19:45:29.447194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:29.444105    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d" podUID="97dda0db-86c4-4e0c-92af-55bb1e85e2f3" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:45:29.859201 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:29.848771    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn" podUID="7f74a4df-72f1-4a7c-a468-76deb45bf8ac" containerName="payload-processing" containerID="cri-o://d0a544627ce45c4561c55d7b5b28443021ff77e457f76641ceab2c5823c450e1" gracePeriod=25
Oct 30 19:45:29.975595 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:29.975557    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj" event={"ID":"662686fd-7d04-48b9-bf7e-adbd30299109","Type":"ContainerStarted","Data":"9d80567b1d0a19ce86df03b37e9753f2599fde6037e6e70d51d5bb73ad5f13e5"}
Oct 30 19:45:29.976140 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:29.976106    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj" event={"ID":"662686fd-7d04-48b9-bf7e-adbd30299109","Type":"ContainerStarted","Data":"98dd9c806afb962b1c5cead0a11ab375f1f2e70eb8d2ab1d971a7dcef1526a19"}
Oct 30 19:45:29.976345 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:29.976160    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"
Oct 30 19:45:29.997953 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:29.997896    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj" podStartSLOduration=2.478116805 podStartE2EDuration="2.997882357s" totalImagesPullingTime="519.765552ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:29.996157616 +0000 UTC m=+2361.737148708" watchObservedRunningTime="2026-10-30 19:45:29.997882357 +0000 UTC m=+2361.738873436"
Oct 30 19:45:30.290131 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:30.290098    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn"
Oct 30 19:45:30.372389 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:30.372339    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7f74a4df-72f1-4a7c-a468-76deb45bf8ac-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f74a4df-72f1-4a7c-a468-76deb45bf8ac-plugins-config-volume\") pod \"7f74a4df-72f1-4a7c-a468-76deb45bf8ac\" (UID: \"7f74a4df-72f1-4a7c-a468-76deb45bf8ac\") "
Oct 30 19:45:30.372611 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:30.372587    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f74a4df-72f1-4a7c-a468-76deb45bf8ac-kube-api-access-mfgg8\" (UniqueName: \"kubernetes.io/projected/7f74a4df-72f1-4a7c-a468-76deb45bf8ac-kube-api-access-mfgg8\") pod \"7f74a4df-72f1-4a7c-a468-76deb45bf8ac\" (UID: \"7f74a4df-72f1-4a7c-a468-76deb45bf8ac\") "
Oct 30 19:45:30.372821 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:30.372788    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f74a4df-72f1-4a7c-a468-76deb45bf8ac-plugins-config-volume" pod "7f74a4df-72f1-4a7c-a468-76deb45bf8ac" (UID: "7f74a4df-72f1-4a7c-a468-76deb45bf8ac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:30.373412 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:30.373382    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f74a4df-72f1-4a7c-a468-76deb45bf8ac-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:30.377007 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:30.376975    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f74a4df-72f1-4a7c-a468-76deb45bf8ac-kube-api-access-mfgg8" pod "7f74a4df-72f1-4a7c-a468-76deb45bf8ac" (UID: "7f74a4df-72f1-4a7c-a468-76deb45bf8ac"). InnerVolumeSpecName "kube-api-access-mfgg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:30.397903 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:30.397878    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd" path="/var/lib/kubelet/pods/1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd/volumes"
Oct 30 19:45:30.474965 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:30.474929    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfgg8\" (UniqueName: \"kubernetes.io/projected/7f74a4df-72f1-4a7c-a468-76deb45bf8ac-kube-api-access-mfgg8\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:30.987332 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:30.987301    4524 generic.go:356] "Generic (PLEG): container finished" podID="7f74a4df-72f1-4a7c-a468-76deb45bf8ac" containerID="d0a544627ce45c4561c55d7b5b28443021ff77e457f76641ceab2c5823c450e1" exitCode=0
Oct 30 19:45:30.987530 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:30.987366    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn"
Oct 30 19:45:30.987530 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:30.987391    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn" event={"ID":"7f74a4df-72f1-4a7c-a468-76deb45bf8ac","Type":"ContainerDied","Data":"d0a544627ce45c4561c55d7b5b28443021ff77e457f76641ceab2c5823c450e1"}
Oct 30 19:45:30.987530 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:30.987466    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn" event={"ID":"7f74a4df-72f1-4a7c-a468-76deb45bf8ac","Type":"ContainerDied","Data":"4e1d27e4d922be4fdf5bfd1fb1b3fb8f4f367608d64c2452f6312ecadae77ded"}
Oct 30 19:45:30.987530 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:30.987498    4524 scope.go:122] "RemoveContainer" containerID="d0a544627ce45c4561c55d7b5b28443021ff77e457f76641ceab2c5823c450e1"
Oct 30 19:45:31.005382 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:31.005344    4524 scope.go:122] "RemoveContainer" containerID="d0a544627ce45c4561c55d7b5b28443021ff77e457f76641ceab2c5823c450e1"
Oct 30 19:45:31.005739 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:45:31.005711    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0a544627ce45c4561c55d7b5b28443021ff77e457f76641ceab2c5823c450e1\": container with ID starting with d0a544627ce45c4561c55d7b5b28443021ff77e457f76641ceab2c5823c450e1 not found: ID does not exist" containerID="d0a544627ce45c4561c55d7b5b28443021ff77e457f76641ceab2c5823c450e1"
Oct 30 19:45:31.005882 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:31.005747    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0a544627ce45c4561c55d7b5b28443021ff77e457f76641ceab2c5823c450e1"} err="failed to get container status \"d0a544627ce45c4561c55d7b5b28443021ff77e457f76641ceab2c5823c450e1\": rpc error: code = NotFound desc = could not find container \"d0a544627ce45c4561c55d7b5b28443021ff77e457f76641ceab2c5823c450e1\": container with ID starting with d0a544627ce45c4561c55d7b5b28443021ff77e457f76641ceab2c5823c450e1 not found: ID does not exist"
Oct 30 19:45:31.013654 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:31.013627    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn"]
Oct 30 19:45:31.017935 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:31.017909    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-1a844e-566cccccb8-2brrn"]
Oct 30 19:45:32.394328 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:32.394297    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7f74a4df-72f1-4a7c-a468-76deb45bf8ac" path="/var/lib/kubelet/pods/7f74a4df-72f1-4a7c-a468-76deb45bf8ac/volumes"
Oct 30 19:45:34.310665 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:34.310626    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"]
Oct 30 19:45:34.311165 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:34.310957    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7" podUID="eec1b428-31dc-4bb4-812e-c575b780ea7f" containerName="maas-api" containerID="cri-o://04ca98c19f1acd4e62bdba8361369ea1cb761265f48b3d0c63baa92cba4055c6" gracePeriod=30
Oct 30 19:45:34.412639 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:45:34.412599    4524 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeec1b428_31dc_4bb4_812e_c575b780ea7f.slice/crio-conmon-04ca98c19f1acd4e62bdba8361369ea1cb761265f48b3d0c63baa92cba4055c6.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 19:45:35.045300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:35.043709    4524 generic.go:356] "Generic (PLEG): container finished" podID="eec1b428-31dc-4bb4-812e-c575b780ea7f" containerID="04ca98c19f1acd4e62bdba8361369ea1cb761265f48b3d0c63baa92cba4055c6" exitCode=0
Oct 30 19:45:35.045300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:35.044032    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7" event={"ID":"eec1b428-31dc-4bb4-812e-c575b780ea7f","Type":"ContainerDied","Data":"04ca98c19f1acd4e62bdba8361369ea1cb761265f48b3d0c63baa92cba4055c6"}
Oct 30 19:45:35.097057 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:35.097032    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"
Oct 30 19:45:35.274479 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:35.270569    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eec1b428-31dc-4bb4-812e-c575b780ea7f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/eec1b428-31dc-4bb4-812e-c575b780ea7f-metrics-certs\") pod \"eec1b428-31dc-4bb4-812e-c575b780ea7f\" (UID: \"eec1b428-31dc-4bb4-812e-c575b780ea7f\") "
Oct 30 19:45:35.274479 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:35.270646    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eec1b428-31dc-4bb4-812e-c575b780ea7f-kube-api-access-vsfh4\" (UniqueName: \"kubernetes.io/projected/eec1b428-31dc-4bb4-812e-c575b780ea7f-kube-api-access-vsfh4\") pod \"eec1b428-31dc-4bb4-812e-c575b780ea7f\" (UID: \"eec1b428-31dc-4bb4-812e-c575b780ea7f\") "
Oct 30 19:45:35.274479 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:35.270736    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eec1b428-31dc-4bb4-812e-c575b780ea7f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eec1b428-31dc-4bb4-812e-c575b780ea7f-openshift-service-ca\") pod \"eec1b428-31dc-4bb4-812e-c575b780ea7f\" (UID: \"eec1b428-31dc-4bb4-812e-c575b780ea7f\") "
Oct 30 19:45:35.274479 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:35.270933    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eec1b428-31dc-4bb4-812e-c575b780ea7f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eec1b428-31dc-4bb4-812e-c575b780ea7f-maas-api-tls\") pod \"eec1b428-31dc-4bb4-812e-c575b780ea7f\" (UID: \"eec1b428-31dc-4bb4-812e-c575b780ea7f\") "
Oct 30 19:45:35.282294 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:35.281953    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eec1b428-31dc-4bb4-812e-c575b780ea7f-openshift-service-ca" pod "eec1b428-31dc-4bb4-812e-c575b780ea7f" (UID: "eec1b428-31dc-4bb4-812e-c575b780ea7f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:35.285832 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:35.285636    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eec1b428-31dc-4bb4-812e-c575b780ea7f-maas-api-tls" pod "eec1b428-31dc-4bb4-812e-c575b780ea7f" (UID: "eec1b428-31dc-4bb4-812e-c575b780ea7f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:35.290060 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:35.290032    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eec1b428-31dc-4bb4-812e-c575b780ea7f-metrics-certs" pod "eec1b428-31dc-4bb4-812e-c575b780ea7f" (UID: "eec1b428-31dc-4bb4-812e-c575b780ea7f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:35.292642 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:35.292616    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eec1b428-31dc-4bb4-812e-c575b780ea7f-kube-api-access-vsfh4" pod "eec1b428-31dc-4bb4-812e-c575b780ea7f" (UID: "eec1b428-31dc-4bb4-812e-c575b780ea7f"). InnerVolumeSpecName "kube-api-access-vsfh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:35.382511 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:35.382469    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eec1b428-31dc-4bb4-812e-c575b780ea7f-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:35.382511 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:35.382511    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsfh4\" (UniqueName: \"kubernetes.io/projected/eec1b428-31dc-4bb4-812e-c575b780ea7f-kube-api-access-vsfh4\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:35.382948 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:35.382531    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eec1b428-31dc-4bb4-812e-c575b780ea7f-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:35.382948 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:35.382544    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eec1b428-31dc-4bb4-812e-c575b780ea7f-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:36.001387 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:36.001219    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"
Oct 30 19:45:36.059327 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:36.059298    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"
Oct 30 19:45:36.059518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:36.059290    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7" event={"ID":"eec1b428-31dc-4bb4-812e-c575b780ea7f","Type":"ContainerDied","Data":"1f6debeb71cf833889d0d10a10954c9cee8b46a4e76dfe22f28a9fc0457f35ce"}
Oct 30 19:45:36.059518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:36.059453    4524 scope.go:122] "RemoveContainer" containerID="04ca98c19f1acd4e62bdba8361369ea1cb761265f48b3d0c63baa92cba4055c6"
Oct 30 19:45:36.092778 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:36.092723    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"]
Oct 30 19:45:36.097830 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:36.097801    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f5613a-869447c5c-ljnc7"]
Oct 30 19:45:36.393669 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:36.393635    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eec1b428-31dc-4bb4-812e-c575b780ea7f" path="/var/lib/kubelet/pods/eec1b428-31dc-4bb4-812e-c575b780ea7f/volumes"
Oct 30 19:45:38.373954 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:38.373906    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g" podUID="b7b95d1a-dbee-45f7-806f-34a1b28a7fcf" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:45:39.413003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:39.412968    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d" podUID="97dda0db-86c4-4e0c-92af-55bb1e85e2f3" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:45:42.472199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.467304    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"]
Oct 30 19:45:42.472199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.468486    4524 cpu_manager.go:414] "containerMap: removing container" podUID="1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd" containerName="maas-api"
Oct 30 19:45:42.472199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.468507    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd" containerName="maas-api"
Oct 30 19:45:42.472199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.468543    4524 cpu_manager.go:414] "containerMap: removing container" podUID="378763b7-0302-43cd-bb31-32f2605d3374" containerName="cleanup"
Oct 30 19:45:42.472199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.468551    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="378763b7-0302-43cd-bb31-32f2605d3374" containerName="cleanup"
Oct 30 19:45:42.472199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.468571    4524 cpu_manager.go:414] "containerMap: removing container" podUID="7f74a4df-72f1-4a7c-a468-76deb45bf8ac" containerName="payload-processing"
Oct 30 19:45:42.472199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.468579    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7f74a4df-72f1-4a7c-a468-76deb45bf8ac" containerName="payload-processing"
Oct 30 19:45:42.472199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.468600    4524 cpu_manager.go:414] "containerMap: removing container" podUID="eec1b428-31dc-4bb4-812e-c575b780ea7f" containerName="maas-api"
Oct 30 19:45:42.472199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.468608    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eec1b428-31dc-4bb4-812e-c575b780ea7f" containerName="maas-api"
Oct 30 19:45:42.472199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.468784    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="7f74a4df-72f1-4a7c-a468-76deb45bf8ac" containerName="payload-processing"
Oct 30 19:45:42.472199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.468797    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="eec1b428-31dc-4bb4-812e-c575b780ea7f" containerName="maas-api"
Oct 30 19:45:42.472199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.468817    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d2a7bd3-5c2c-4e75-aac9-679a0f7c8ebd" containerName="maas-api"
Oct 30 19:45:42.472199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.468830    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="378763b7-0302-43cd-bb31-32f2605d3374" containerName="cleanup"
Oct 30 19:45:42.529340 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.510712    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.604197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.584200    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-c2b092-openshift-default-dockercfg-8hfcl\""
Oct 30 19:45:42.621190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.617230    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-token\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.621190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.617361    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-workload-certs\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.621190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.617394    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e3ef760-19e1-45bc-b806-57d79f5082db-istiod-ca-cert\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.621190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.617428    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-data\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.621190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.617478    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfkk9\" (UniqueName: \"kubernetes.io/projected/1e3ef760-19e1-45bc-b806-57d79f5082db-kube-api-access-pfkk9\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.621190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.617511    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-workload-socket\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.621190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.617559    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-credential-socket\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.621190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.617748    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-podinfo\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.621190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.617841    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-envoy\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.718196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.710511    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"]
Oct 30 19:45:42.728232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.719358    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-workload-certs\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.728232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.719402    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e3ef760-19e1-45bc-b806-57d79f5082db-istiod-ca-cert\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.728232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.719441    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-data\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.728232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.719495    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfkk9\" (UniqueName: \"kubernetes.io/projected/1e3ef760-19e1-45bc-b806-57d79f5082db-kube-api-access-pfkk9\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.728232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.719534    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-workload-socket\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.728232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.719585    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-credential-socket\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.728232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.719783    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-podinfo\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.728232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.719879    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-envoy\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.728232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.719924    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-token\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.728232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.720905    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-workload-certs\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.728232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.721581    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e3ef760-19e1-45bc-b806-57d79f5082db-istiod-ca-cert\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.728232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.721821    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-data\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.728232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.724158    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-workload-socket\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.728232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.724446    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-credential-socket\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.770980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.770012    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfkk9\" (UniqueName: \"kubernetes.io/projected/1e3ef760-19e1-45bc-b806-57d79f5082db-kube-api-access-pfkk9\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.775986 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.775958    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-token\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.784824 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.784789    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-podinfo\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:42.788752 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:42.787079    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-envoy\") pod \"e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") " pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:43.030985 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:43.030635    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:43.345841 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:45:43.345808    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3ef760_19e1_45bc_b806_57d79f5082db.slice/crio-eb5aee72ae4d2c4d2679baaf5608fb3fce90b142121d6c0addff83acebceaa54 WatchSource:0}: Error finding container eb5aee72ae4d2c4d2679baaf5608fb3fce90b142121d6c0addff83acebceaa54: Status 404 returned error can't find the container with id eb5aee72ae4d2c4d2679baaf5608fb3fce90b142121d6c0addff83acebceaa54
Oct 30 19:45:43.346187 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:43.346153    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"]
Oct 30 19:45:43.348619 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:43.348585    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:45:43.348707 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:43.348648    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:45:43.348707 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:43.348677    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:45:44.220949 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:44.220907    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4" event={"ID":"1e3ef760-19e1-45bc-b806-57d79f5082db","Type":"ContainerStarted","Data":"3dcd0747cf48ef6b9a8c3a58428d58adc28f9995be16d70bc4709ade02733049"}
Oct 30 19:45:44.221501 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:44.220976    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4" event={"ID":"1e3ef760-19e1-45bc-b806-57d79f5082db","Type":"ContainerStarted","Data":"eb5aee72ae4d2c4d2679baaf5608fb3fce90b142121d6c0addff83acebceaa54"}
Oct 30 19:45:44.320440 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:44.319732    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4" podStartSLOduration=2.319711878 podStartE2EDuration="2.319711878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:44.312704006 +0000 UTC m=+2376.053695098" watchObservedRunningTime="2026-10-30 19:45:44.319711878 +0000 UTC m=+2376.060702955"
Oct 30 19:45:45.031130 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:45.031095    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:45.032552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:45.032527    4524 patch_prober.go:29] interesting pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused" start-of-body=
Oct 30 19:45:45.032697 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:45.032580    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4" podUID="1e3ef760-19e1-45bc-b806-57d79f5082db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused"
Oct 30 19:45:45.632593 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:45:45.632559    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:45:46.031448 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:46.031366    4524 patch_prober.go:29] interesting pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused" start-of-body=
Oct 30 19:45:46.031448 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:46.031425    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4" podUID="1e3ef760-19e1-45bc-b806-57d79f5082db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused"
Oct 30 19:45:47.032328 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:47.032292    4524 patch_prober.go:29] interesting pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused" start-of-body=
Oct 30 19:45:47.032908 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:47.032876    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4" podUID="1e3ef760-19e1-45bc-b806-57d79f5082db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused"
Oct 30 19:45:48.031070 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:48.031037    4524 patch_prober.go:29] interesting pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused" start-of-body=
Oct 30 19:45:48.031319 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:48.031102    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4" podUID="1e3ef760-19e1-45bc-b806-57d79f5082db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused"
Oct 30 19:45:48.407333 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:48.407277    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g" podUID="b7b95d1a-dbee-45f7-806f-34a1b28a7fcf" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:45:49.033682 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:49.033645    4524 patch_prober.go:29] interesting pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused" start-of-body=
Oct 30 19:45:49.033989 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:49.033958    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4" podUID="1e3ef760-19e1-45bc-b806-57d79f5082db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused"
Oct 30 19:45:49.412979 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:45:49.412571    4524 logging.go:55] [core] [Channel #7240 SubChannel #7241]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.109:50052", ServerName: "10.133.0.109:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.109:50052: connect: connection refused"
Oct 30 19:45:49.816202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:49.802577    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh"]
Oct 30 19:45:49.923073 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:49.907409    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d"
Oct 30 19:45:49.960652 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:49.960473    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97dda0db-86c4-4e0c-92af-55bb1e85e2f3-kube-api-access-kfxh9\" (UniqueName: \"kubernetes.io/projected/97dda0db-86c4-4e0c-92af-55bb1e85e2f3-kube-api-access-kfxh9\") pod \"97dda0db-86c4-4e0c-92af-55bb1e85e2f3\" (UID: \"97dda0db-86c4-4e0c-92af-55bb1e85e2f3\") "
Oct 30 19:45:49.960652 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:49.960567    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97dda0db-86c4-4e0c-92af-55bb1e85e2f3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97dda0db-86c4-4e0c-92af-55bb1e85e2f3-plugins-config-volume\") pod \"97dda0db-86c4-4e0c-92af-55bb1e85e2f3\" (UID: \"97dda0db-86c4-4e0c-92af-55bb1e85e2f3\") "
Oct 30 19:45:49.966933 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:49.961559    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97dda0db-86c4-4e0c-92af-55bb1e85e2f3-plugins-config-volume" pod "97dda0db-86c4-4e0c-92af-55bb1e85e2f3" (UID: "97dda0db-86c4-4e0c-92af-55bb1e85e2f3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:49.987190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:49.967354    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97dda0db-86c4-4e0c-92af-55bb1e85e2f3-kube-api-access-kfxh9" pod "97dda0db-86c4-4e0c-92af-55bb1e85e2f3" (UID: "97dda0db-86c4-4e0c-92af-55bb1e85e2f3"). InnerVolumeSpecName "kube-api-access-kfxh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:50.020345 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.020314    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g"
Oct 30 19:45:50.031849 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.031815    4524 patch_prober.go:29] interesting pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused" start-of-body=
Oct 30 19:45:50.032071 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.032037    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4" podUID="1e3ef760-19e1-45bc-b806-57d79f5082db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.181:15021/healthz/ready\": dial tcp 10.133.0.181:15021: connect: connection refused"
Oct 30 19:45:50.151660 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.149650    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b7b95d1a-dbee-45f7-806f-34a1b28a7fcf-kube-api-access-9zpsd\" (UniqueName: \"kubernetes.io/projected/b7b95d1a-dbee-45f7-806f-34a1b28a7fcf-kube-api-access-9zpsd\") pod \"b7b95d1a-dbee-45f7-806f-34a1b28a7fcf\" (UID: \"b7b95d1a-dbee-45f7-806f-34a1b28a7fcf\") "
Oct 30 19:45:50.151660 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.149729    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b7b95d1a-dbee-45f7-806f-34a1b28a7fcf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7b95d1a-dbee-45f7-806f-34a1b28a7fcf-plugins-config-volume\") pod \"b7b95d1a-dbee-45f7-806f-34a1b28a7fcf\" (UID: \"b7b95d1a-dbee-45f7-806f-34a1b28a7fcf\") "
Oct 30 19:45:50.154522 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.154487    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kfxh9\" (UniqueName: \"kubernetes.io/projected/97dda0db-86c4-4e0c-92af-55bb1e85e2f3-kube-api-access-kfxh9\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:50.154706 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.154688    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/97dda0db-86c4-4e0c-92af-55bb1e85e2f3-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:50.155216 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.155189    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7b95d1a-dbee-45f7-806f-34a1b28a7fcf-plugins-config-volume" pod "b7b95d1a-dbee-45f7-806f-34a1b28a7fcf" (UID: "b7b95d1a-dbee-45f7-806f-34a1b28a7fcf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:50.278233 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.270657    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7b95d1a-dbee-45f7-806f-34a1b28a7fcf-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:50.297192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.290798    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7b95d1a-dbee-45f7-806f-34a1b28a7fcf-kube-api-access-9zpsd" pod "b7b95d1a-dbee-45f7-806f-34a1b28a7fcf" (UID: "b7b95d1a-dbee-45f7-806f-34a1b28a7fcf"). InnerVolumeSpecName "kube-api-access-9zpsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:50.362193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.358932    4524 generic.go:356] "Generic (PLEG): container finished" podID="b7b95d1a-dbee-45f7-806f-34a1b28a7fcf" containerID="f166492b87b0df6fa0e667b36463add7d7adb6a242a321d3da61230b1720a3e8" exitCode=0
Oct 30 19:45:50.362193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.359153    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g" event={"ID":"b7b95d1a-dbee-45f7-806f-34a1b28a7fcf","Type":"ContainerDied","Data":"f166492b87b0df6fa0e667b36463add7d7adb6a242a321d3da61230b1720a3e8"}
Oct 30 19:45:50.362193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.359226    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g" event={"ID":"b7b95d1a-dbee-45f7-806f-34a1b28a7fcf","Type":"ContainerDied","Data":"0fcf2a93414f1ba11fb6b081e0982f52f3be92c076abcbc87f83d6080653d312"}
Oct 30 19:45:50.362193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.359262    4524 scope.go:122] "RemoveContainer" containerID="f166492b87b0df6fa0e667b36463add7d7adb6a242a321d3da61230b1720a3e8"
Oct 30 19:45:50.362193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.359421    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g"
Oct 30 19:45:50.381222 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.373219    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zpsd\" (UniqueName: \"kubernetes.io/projected/b7b95d1a-dbee-45f7-806f-34a1b28a7fcf-kube-api-access-9zpsd\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:50.381222 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.377055    4524 generic.go:356] "Generic (PLEG): container finished" podID="97dda0db-86c4-4e0c-92af-55bb1e85e2f3" containerID="f76675191dc14101d26001b1232f3c4a1660d1b752ab49f1fa3d55e4072332be" exitCode=0
Oct 30 19:45:50.381222 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.377210    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d" event={"ID":"97dda0db-86c4-4e0c-92af-55bb1e85e2f3","Type":"ContainerDied","Data":"f76675191dc14101d26001b1232f3c4a1660d1b752ab49f1fa3d55e4072332be"}
Oct 30 19:45:50.381222 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.377284    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d" event={"ID":"97dda0db-86c4-4e0c-92af-55bb1e85e2f3","Type":"ContainerDied","Data":"ea42ccd1942a73aede11d83c4d79d3412a8aa38890f1fc835ebf449cffcc439d"}
Oct 30 19:45:50.381222 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.377389    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d"
Oct 30 19:45:50.415395 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.415000    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d" podUID="97dda0db-86c4-4e0c-92af-55bb1e85e2f3" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.109:50052\" within 1s: context deadline exceeded"
Oct 30 19:45:50.415931 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:45:50.415405    4524 logging.go:55] [core] [Channel #7240 SubChannel #7241]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.109:50052", ServerName: "10.133.0.109:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.109:50052: operation was canceled"
Oct 30 19:45:50.415931 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.415584    4524 scope.go:122] "RemoveContainer" containerID="f166492b87b0df6fa0e667b36463add7d7adb6a242a321d3da61230b1720a3e8"
Oct 30 19:45:50.416067 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:45:50.415949    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f166492b87b0df6fa0e667b36463add7d7adb6a242a321d3da61230b1720a3e8\": container with ID starting with f166492b87b0df6fa0e667b36463add7d7adb6a242a321d3da61230b1720a3e8 not found: ID does not exist" containerID="f166492b87b0df6fa0e667b36463add7d7adb6a242a321d3da61230b1720a3e8"
Oct 30 19:45:50.416067 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.415983    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f166492b87b0df6fa0e667b36463add7d7adb6a242a321d3da61230b1720a3e8"} err="failed to get container status \"f166492b87b0df6fa0e667b36463add7d7adb6a242a321d3da61230b1720a3e8\": rpc error: code = NotFound desc = could not find container \"f166492b87b0df6fa0e667b36463add7d7adb6a242a321d3da61230b1720a3e8\": container with ID starting with f166492b87b0df6fa0e667b36463add7d7adb6a242a321d3da61230b1720a3e8 not found: ID does not exist"
Oct 30 19:45:50.416067 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.416005    4524 scope.go:122] "RemoveContainer" containerID="f76675191dc14101d26001b1232f3c4a1660d1b752ab49f1fa3d55e4072332be"
Oct 30 19:45:50.425192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.417589    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d"
Oct 30 19:45:50.425192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.423218    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g"]
Oct 30 19:45:50.433531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.433502    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-618d93-84cfb77695-6fv8g"]
Oct 30 19:45:50.434416 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.434397    4524 scope.go:122] "RemoveContainer" containerID="f76675191dc14101d26001b1232f3c4a1660d1b752ab49f1fa3d55e4072332be"
Oct 30 19:45:50.434721 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:45:50.434694    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f76675191dc14101d26001b1232f3c4a1660d1b752ab49f1fa3d55e4072332be\": container with ID starting with f76675191dc14101d26001b1232f3c4a1660d1b752ab49f1fa3d55e4072332be not found: ID does not exist" containerID="f76675191dc14101d26001b1232f3c4a1660d1b752ab49f1fa3d55e4072332be"
Oct 30 19:45:50.434877 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.434731    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f76675191dc14101d26001b1232f3c4a1660d1b752ab49f1fa3d55e4072332be"} err="failed to get container status \"f76675191dc14101d26001b1232f3c4a1660d1b752ab49f1fa3d55e4072332be\": rpc error: code = NotFound desc = could not find container \"f76675191dc14101d26001b1232f3c4a1660d1b752ab49f1fa3d55e4072332be\": container with ID starting with f76675191dc14101d26001b1232f3c4a1660d1b752ab49f1fa3d55e4072332be not found: ID does not exist"
Oct 30 19:45:50.440638 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.440614    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d"]
Oct 30 19:45:50.448838 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:50.448814    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-618d93-68dd6997f6-tdm7d"]
Oct 30 19:45:51.064248 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:51.064219    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:51.065881 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:51.065858    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:51.072058 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:51.072039    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:45:52.404098 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:52.404051    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97dda0db-86c4-4e0c-92af-55bb1e85e2f3" path="/var/lib/kubelet/pods/97dda0db-86c4-4e0c-92af-55bb1e85e2f3/volumes"
Oct 30 19:45:52.404623 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:52.404603    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b7b95d1a-dbee-45f7-806f-34a1b28a7fcf" path="/var/lib/kubelet/pods/b7b95d1a-dbee-45f7-806f-34a1b28a7fcf/volumes"
Oct 30 19:45:54.658976 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:54.658938    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87"]
Oct 30 19:45:54.667588 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:54.667554    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk"]
Oct 30 19:45:54.803321 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:54.803279    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh" podUID="776185e2-f0f0-4a7e-b4ee-3d5f2e429f46" containerName="payload-pre-processing" containerID="cri-o://52427cdffc62476c567ede3a700a3c34a3e8291c6ff22bb0e4f213e51d2e520a" gracePeriod=25
Oct 30 19:45:55.391512 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:55.391480    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh"
Oct 30 19:45:55.407704 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:55.407671    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"]
Oct 30 19:45:55.407908 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:55.407881    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47" podUID="657a30cd-92e3-449c-971b-b3e0ffe929cc" containerName="maas-api" containerID="cri-o://42b2df0a5bc427e6877b5d6175ff419515a9cb1d18b46b59f873154366555914" gracePeriod=30
Oct 30 19:45:55.484336 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:55.484299    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/776185e2-f0f0-4a7e-b4ee-3d5f2e429f46-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/776185e2-f0f0-4a7e-b4ee-3d5f2e429f46-plugins-config-volume\") pod \"776185e2-f0f0-4a7e-b4ee-3d5f2e429f46\" (UID: \"776185e2-f0f0-4a7e-b4ee-3d5f2e429f46\") "
Oct 30 19:45:55.484648 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:55.484623    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/776185e2-f0f0-4a7e-b4ee-3d5f2e429f46-kube-api-access-gbn4t\" (UniqueName: \"kubernetes.io/projected/776185e2-f0f0-4a7e-b4ee-3d5f2e429f46-kube-api-access-gbn4t\") pod \"776185e2-f0f0-4a7e-b4ee-3d5f2e429f46\" (UID: \"776185e2-f0f0-4a7e-b4ee-3d5f2e429f46\") "
Oct 30 19:45:55.489877 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:55.486336    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/776185e2-f0f0-4a7e-b4ee-3d5f2e429f46-plugins-config-volume" pod "776185e2-f0f0-4a7e-b4ee-3d5f2e429f46" (UID: "776185e2-f0f0-4a7e-b4ee-3d5f2e429f46"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:55.491955 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:55.491930    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/776185e2-f0f0-4a7e-b4ee-3d5f2e429f46-kube-api-access-gbn4t" pod "776185e2-f0f0-4a7e-b4ee-3d5f2e429f46" (UID: "776185e2-f0f0-4a7e-b4ee-3d5f2e429f46"). InnerVolumeSpecName "kube-api-access-gbn4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:55.588221 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:55.588184    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/776185e2-f0f0-4a7e-b4ee-3d5f2e429f46-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.588221 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:55.588215    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbn4t\" (UniqueName: \"kubernetes.io/projected/776185e2-f0f0-4a7e-b4ee-3d5f2e429f46-kube-api-access-gbn4t\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:55.963196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:55.955487    4524 generic.go:356] "Generic (PLEG): container finished" podID="657a30cd-92e3-449c-971b-b3e0ffe929cc" containerID="42b2df0a5bc427e6877b5d6175ff419515a9cb1d18b46b59f873154366555914" exitCode=0
Oct 30 19:45:55.963196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:55.955866    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47" event={"ID":"657a30cd-92e3-449c-971b-b3e0ffe929cc","Type":"ContainerDied","Data":"42b2df0a5bc427e6877b5d6175ff419515a9cb1d18b46b59f873154366555914"}
Oct 30 19:45:55.963196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:55.957779    4524 generic.go:356] "Generic (PLEG): container finished" podID="776185e2-f0f0-4a7e-b4ee-3d5f2e429f46" containerID="52427cdffc62476c567ede3a700a3c34a3e8291c6ff22bb0e4f213e51d2e520a" exitCode=0
Oct 30 19:45:55.963196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:55.957822    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh" event={"ID":"776185e2-f0f0-4a7e-b4ee-3d5f2e429f46","Type":"ContainerDied","Data":"52427cdffc62476c567ede3a700a3c34a3e8291c6ff22bb0e4f213e51d2e520a"}
Oct 30 19:45:55.963196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:55.957863    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh" event={"ID":"776185e2-f0f0-4a7e-b4ee-3d5f2e429f46","Type":"ContainerDied","Data":"c35afac143b070c9b4878acdbb6fa5d75429752ce5b6116bf96d649c5cae48b9"}
Oct 30 19:45:55.963196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:55.957891    4524 scope.go:122] "RemoveContainer" containerID="52427cdffc62476c567ede3a700a3c34a3e8291c6ff22bb0e4f213e51d2e520a"
Oct 30 19:45:55.963196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:55.958045    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh"
Oct 30 19:45:56.104422 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.104378    4524 scope.go:122] "RemoveContainer" containerID="52427cdffc62476c567ede3a700a3c34a3e8291c6ff22bb0e4f213e51d2e520a"
Oct 30 19:45:56.104779 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:45:56.104750    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52427cdffc62476c567ede3a700a3c34a3e8291c6ff22bb0e4f213e51d2e520a\": container with ID starting with 52427cdffc62476c567ede3a700a3c34a3e8291c6ff22bb0e4f213e51d2e520a not found: ID does not exist" containerID="52427cdffc62476c567ede3a700a3c34a3e8291c6ff22bb0e4f213e51d2e520a"
Oct 30 19:45:56.104882 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.104790    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52427cdffc62476c567ede3a700a3c34a3e8291c6ff22bb0e4f213e51d2e520a"} err="failed to get container status \"52427cdffc62476c567ede3a700a3c34a3e8291c6ff22bb0e4f213e51d2e520a\": rpc error: code = NotFound desc = could not find container \"52427cdffc62476c567ede3a700a3c34a3e8291c6ff22bb0e4f213e51d2e520a\": container with ID starting with 52427cdffc62476c567ede3a700a3c34a3e8291c6ff22bb0e4f213e51d2e520a not found: ID does not exist"
Oct 30 19:45:56.107662 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.107629    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"
Oct 30 19:45:56.114729 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.114701    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh"]
Oct 30 19:45:56.119256 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.119233    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-826518-6996d58587-xxsxh"]
Oct 30 19:45:56.253896 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.253787    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/657a30cd-92e3-449c-971b-b3e0ffe929cc-kube-api-access-5f7zm\" (UniqueName: \"kubernetes.io/projected/657a30cd-92e3-449c-971b-b3e0ffe929cc-kube-api-access-5f7zm\") pod \"657a30cd-92e3-449c-971b-b3e0ffe929cc\" (UID: \"657a30cd-92e3-449c-971b-b3e0ffe929cc\") "
Oct 30 19:45:56.254066 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.253890    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/657a30cd-92e3-449c-971b-b3e0ffe929cc-metrics-certs\" (UniqueName: \"kubernetes.io/secret/657a30cd-92e3-449c-971b-b3e0ffe929cc-metrics-certs\") pod \"657a30cd-92e3-449c-971b-b3e0ffe929cc\" (UID: \"657a30cd-92e3-449c-971b-b3e0ffe929cc\") "
Oct 30 19:45:56.254066 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.254057    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/657a30cd-92e3-449c-971b-b3e0ffe929cc-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/657a30cd-92e3-449c-971b-b3e0ffe929cc-maas-api-tls\") pod \"657a30cd-92e3-449c-971b-b3e0ffe929cc\" (UID: \"657a30cd-92e3-449c-971b-b3e0ffe929cc\") "
Oct 30 19:45:56.254264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.254089    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/657a30cd-92e3-449c-971b-b3e0ffe929cc-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/657a30cd-92e3-449c-971b-b3e0ffe929cc-openshift-service-ca\") pod \"657a30cd-92e3-449c-971b-b3e0ffe929cc\" (UID: \"657a30cd-92e3-449c-971b-b3e0ffe929cc\") "
Oct 30 19:45:56.261371 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.255801    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/657a30cd-92e3-449c-971b-b3e0ffe929cc-openshift-service-ca" pod "657a30cd-92e3-449c-971b-b3e0ffe929cc" (UID: "657a30cd-92e3-449c-971b-b3e0ffe929cc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:56.261371 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.258628    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/657a30cd-92e3-449c-971b-b3e0ffe929cc-maas-api-tls" pod "657a30cd-92e3-449c-971b-b3e0ffe929cc" (UID: "657a30cd-92e3-449c-971b-b3e0ffe929cc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:56.261371 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.259507    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/657a30cd-92e3-449c-971b-b3e0ffe929cc-metrics-certs" pod "657a30cd-92e3-449c-971b-b3e0ffe929cc" (UID: "657a30cd-92e3-449c-971b-b3e0ffe929cc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:45:56.261577 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.261368    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/657a30cd-92e3-449c-971b-b3e0ffe929cc-kube-api-access-5f7zm" pod "657a30cd-92e3-449c-971b-b3e0ffe929cc" (UID: "657a30cd-92e3-449c-971b-b3e0ffe929cc"). InnerVolumeSpecName "kube-api-access-5f7zm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:56.355859 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.355826    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5f7zm\" (UniqueName: \"kubernetes.io/projected/657a30cd-92e3-449c-971b-b3e0ffe929cc-kube-api-access-5f7zm\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:56.355859 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.355853    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/657a30cd-92e3-449c-971b-b3e0ffe929cc-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:56.355859 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.355864    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/657a30cd-92e3-449c-971b-b3e0ffe929cc-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:56.356080 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.355874    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/657a30cd-92e3-449c-971b-b3e0ffe929cc-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:56.395984 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.395948    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="776185e2-f0f0-4a7e-b4ee-3d5f2e429f46" path="/var/lib/kubelet/pods/776185e2-f0f0-4a7e-b4ee-3d5f2e429f46/volumes"
Oct 30 19:45:56.968393 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.968354    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47" event={"ID":"657a30cd-92e3-449c-971b-b3e0ffe929cc","Type":"ContainerDied","Data":"68fb54edf42e2f6b0356decbe6f22755506032c01ffa2b91ce1088f86f241ec6"}
Oct 30 19:45:56.968393 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.968386    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"
Oct 30 19:45:56.968924 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.968428    4524 scope.go:122] "RemoveContainer" containerID="42b2df0a5bc427e6877b5d6175ff419515a9cb1d18b46b59f873154366555914"
Oct 30 19:45:56.991161 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.991137    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"]
Oct 30 19:45:56.995661 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:56.995639    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-e3586f-7c577fcf5b-l9t47"]
Oct 30 19:45:58.093612 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.093580    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"]
Oct 30 19:45:58.094922 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.094889    4524 cpu_manager.go:414] "containerMap: removing container" podUID="776185e2-f0f0-4a7e-b4ee-3d5f2e429f46" containerName="payload-pre-processing"
Oct 30 19:45:58.095065 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.094943    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="776185e2-f0f0-4a7e-b4ee-3d5f2e429f46" containerName="payload-pre-processing"
Oct 30 19:45:58.095065 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.094977    4524 cpu_manager.go:414] "containerMap: removing container" podUID="97dda0db-86c4-4e0c-92af-55bb1e85e2f3" containerName="payload-processing"
Oct 30 19:45:58.095065 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.094985    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97dda0db-86c4-4e0c-92af-55bb1e85e2f3" containerName="payload-processing"
Oct 30 19:45:58.095326 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.095161    4524 cpu_manager.go:414] "containerMap: removing container" podUID="657a30cd-92e3-449c-971b-b3e0ffe929cc" containerName="maas-api"
Oct 30 19:45:58.095326 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.095190    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="657a30cd-92e3-449c-971b-b3e0ffe929cc" containerName="maas-api"
Oct 30 19:45:58.095326 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.095216    4524 cpu_manager.go:414] "containerMap: removing container" podUID="b7b95d1a-dbee-45f7-806f-34a1b28a7fcf" containerName="payload-pre-processing"
Oct 30 19:45:58.095326 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.095225    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b7b95d1a-dbee-45f7-806f-34a1b28a7fcf" containerName="payload-pre-processing"
Oct 30 19:45:58.095550 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.095419    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="657a30cd-92e3-449c-971b-b3e0ffe929cc" containerName="maas-api"
Oct 30 19:45:58.095550 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.095435    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="b7b95d1a-dbee-45f7-806f-34a1b28a7fcf" containerName="payload-pre-processing"
Oct 30 19:45:58.095550 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.095451    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="776185e2-f0f0-4a7e-b4ee-3d5f2e429f46" containerName="payload-pre-processing"
Oct 30 19:45:58.095550 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.095464    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="97dda0db-86c4-4e0c-92af-55bb1e85e2f3" containerName="payload-processing"
Oct 30 19:45:58.101527 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.101504    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.117384 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.117360    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-1a844e-openshift-default-dockercfg-465qc\""
Oct 30 19:45:58.124440 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.124416    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"]
Oct 30 19:45:58.186150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.186111    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-istio-data\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.186315 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.186272    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-credential-socket\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.186454 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.186431    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-workload-socket\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.186505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.186487    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c074845-90e7-46eb-bed0-9360969cf36e-istiod-ca-cert\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.186610 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.186595    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v67jp\" (UniqueName: \"kubernetes.io/projected/5c074845-90e7-46eb-bed0-9360969cf36e-kube-api-access-v67jp\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.186708 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.186693    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-workload-certs\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.186756 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.186743    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-istio-envoy\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.186851 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.186829    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c074845-90e7-46eb-bed0-9360969cf36e-istio-podinfo\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.186890 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.186874    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c074845-90e7-46eb-bed0-9360969cf36e-istio-token\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.288399 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.288354    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-istio-envoy\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.288588 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.288422    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c074845-90e7-46eb-bed0-9360969cf36e-istio-podinfo\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.288588 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.288466    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c074845-90e7-46eb-bed0-9360969cf36e-istio-token\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.288588 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.288545    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-istio-data\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.288776 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.288627    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-credential-socket\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.288776 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.288713    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-workload-socket\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.288776 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.288752    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c074845-90e7-46eb-bed0-9360969cf36e-istiod-ca-cert\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.288925 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.288837    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v67jp\" (UniqueName: \"kubernetes.io/projected/5c074845-90e7-46eb-bed0-9360969cf36e-kube-api-access-v67jp\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.289929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.289258    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-workload-certs\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.289929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.289291    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-credential-socket\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.289929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.289596    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-workload-socket\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.289929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.289870    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-workload-certs\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.290277 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.290053    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c074845-90e7-46eb-bed0-9360969cf36e-istiod-ca-cert\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.290277 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.290196    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-istio-data\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.291838 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.291733    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-istio-envoy\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.292030 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.292010    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c074845-90e7-46eb-bed0-9360969cf36e-istio-podinfo\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.299953 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.299928    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c074845-90e7-46eb-bed0-9360969cf36e-istio-token\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.305452 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.305429    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v67jp\" (UniqueName: \"kubernetes.io/projected/5c074845-90e7-46eb-bed0-9360969cf36e-kube-api-access-v67jp\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.396492 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.396407    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="657a30cd-92e3-449c-971b-b3e0ffe929cc" path="/var/lib/kubelet/pods/657a30cd-92e3-449c-971b-b3e0ffe929cc/volumes"
Oct 30 19:45:58.425803 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.425773    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:58.721874 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:45:58.721839    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c074845_90e7_46eb_bed0_9360969cf36e.slice/crio-c016f2624d85db53edbd30f7830f88fa1438e5c6bff00c5f66794d282e125226 WatchSource:0}: Error finding container c016f2624d85db53edbd30f7830f88fa1438e5c6bff00c5f66794d282e125226: Status 404 returned error can't find the container with id c016f2624d85db53edbd30f7830f88fa1438e5c6bff00c5f66794d282e125226
Oct 30 19:45:58.724823 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.724757    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:45:58.724919 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.724859    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:45:58.724919 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.724894    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:45:58.725885 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.725858    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"]
Oct 30 19:45:58.995521 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.995425    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79" event={"ID":"5c074845-90e7-46eb-bed0-9360969cf36e","Type":"ContainerStarted","Data":"2d20b11e092e9e4fa2d099d2c88e2233190cad23d1cf576c7e8ec8bd4e2ccdad"}
Oct 30 19:45:58.995521 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:58.995500    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79" event={"ID":"5c074845-90e7-46eb-bed0-9360969cf36e","Type":"ContainerStarted","Data":"c016f2624d85db53edbd30f7830f88fa1438e5c6bff00c5f66794d282e125226"}
Oct 30 19:45:59.018143 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:59.018081    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79" podStartSLOduration=1.018063675 podStartE2EDuration="1.018063675s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:59.016533392 +0000 UTC m=+2390.757524472" watchObservedRunningTime="2026-10-30 19:45:59.018063675 +0000 UTC m=+2390.759054755"
Oct 30 19:45:59.430690 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:59.430657    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:45:59.432793 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:59.432764    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused" start-of-body=
Oct 30 19:45:59.432944 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:59.432819    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79" podUID="5c074845-90e7-46eb-bed0-9360969cf36e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused"
Oct 30 19:45:59.659579 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:59.659505    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87" podUID="1c50188a-2ca9-4063-9092-9fdd095582aa" containerName="payload-processing" containerID="cri-o://5fe9601fadd8cf475b64efba769a18e4dac62beaa7e097d3cb983176d0c33000" gracePeriod=25
Oct 30 19:45:59.671526 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:45:59.671323    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk" podUID="533f548c-cf0b-4bd7-9fed-2d9bde66ec95" containerName="payload-pre-processing" containerID="cri-o://82e3e1defc32af005ce07a40e50bc0289314db1694f99cc9924d6ed98b6099d0" gracePeriod=25
Oct 30 19:46:00.211027 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:00.210992    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk" podUID="533f548c-cf0b-4bd7-9fed-2d9bde66ec95" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:00.426296 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:00.426265    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused" start-of-body=
Oct 30 19:46:00.426449 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:00.426317    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79" podUID="5c074845-90e7-46eb-bed0-9360969cf36e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused"
Oct 30 19:46:01.237494 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:01.237454    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87" podUID="1c50188a-2ca9-4063-9092-9fdd095582aa" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:01.426452 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:01.426422    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused" start-of-body=
Oct 30 19:46:01.426624 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:01.426485    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79" podUID="5c074845-90e7-46eb-bed0-9360969cf36e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused"
Oct 30 19:46:01.835619 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:01.833672    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"]
Oct 30 19:46:01.839781 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:01.838864    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:01.844669 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:01.844330    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-c0a27e-openshift-default-dockercfg-99cb4\""
Oct 30 19:46:01.844801 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:01.844723    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"]
Oct 30 19:46:01.992234 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:01.946539    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-workload-certs\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.080194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.062809    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-istio-data\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.080194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.062900    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-workload-socket\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.080194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.062930    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-credential-socket\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.080194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.062966    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d49fd92-802a-402c-8098-414925be3659-istio-token\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.080194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.063048    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2mbh\" (UniqueName: \"kubernetes.io/projected/0d49fd92-802a-402c-8098-414925be3659-kube-api-access-c2mbh\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.080194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.063128    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-istio-envoy\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.080194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.063261    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d49fd92-802a-402c-8098-414925be3659-istio-podinfo\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.080194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.063302    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d49fd92-802a-402c-8098-414925be3659-istiod-ca-cert\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.170120 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.169330    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-workload-socket\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.170120 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.169377    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-credential-socket\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.170120 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.169415    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d49fd92-802a-402c-8098-414925be3659-istio-token\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.170120 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.169493    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2mbh\" (UniqueName: \"kubernetes.io/projected/0d49fd92-802a-402c-8098-414925be3659-kube-api-access-c2mbh\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.170120 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.169563    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-istio-envoy\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.170120 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.169666    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d49fd92-802a-402c-8098-414925be3659-istio-podinfo\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.170120 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.169703    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d49fd92-802a-402c-8098-414925be3659-istiod-ca-cert\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.170120 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.169883    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-workload-certs\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.170705 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.170190    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-istio-data\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.170705 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.170695    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-istio-data\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.185008 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.171793    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-workload-socket\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.185008 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.172023    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-credential-socket\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.326506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.326473    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-workload-certs\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.338330 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.338292    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d49fd92-802a-402c-8098-414925be3659-istiod-ca-cert\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.344450 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.344416    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d49fd92-802a-402c-8098-414925be3659-istio-token\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.348083 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.348055    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d49fd92-802a-402c-8098-414925be3659-istio-podinfo\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.353937 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.353886    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2mbh\" (UniqueName: \"kubernetes.io/projected/0d49fd92-802a-402c-8098-414925be3659-kube-api-access-c2mbh\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.360093 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.360000    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-istio-envoy\") pod \"e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") " pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.427200 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.427084    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused" start-of-body=
Oct 30 19:46:02.427200 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.427162    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79" podUID="5c074845-90e7-46eb-bed0-9360969cf36e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused"
Oct 30 19:46:02.497305 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.495952    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:02.985193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.980506    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"]
Oct 30 19:46:02.985193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.981138    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:46:02.985193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.981245    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:46:02.985193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:02.981290    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:46:03.187216 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:03.167391    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km" event={"ID":"0d49fd92-802a-402c-8098-414925be3659","Type":"ContainerStarted","Data":"2c435bc51175861f0ebeb67308063a1b9d99ab6bc75e47d00b174f8bf3155eab"}
Oct 30 19:46:03.426664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:03.426625    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused" start-of-body=
Oct 30 19:46:03.427132 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:03.426683    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79" podUID="5c074845-90e7-46eb-bed0-9360969cf36e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused"
Oct 30 19:46:04.204502 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:04.204460    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km" event={"ID":"0d49fd92-802a-402c-8098-414925be3659","Type":"ContainerStarted","Data":"cb9a95485bf80337be1c2764e89bc5f1000c889a2517d4072846ecd741ba4595"}
Oct 30 19:46:04.231551 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:04.231495    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km" podStartSLOduration=3.231481606 podStartE2EDuration="3.231481606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:46:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:46:04.226567097 +0000 UTC m=+2395.967558179" watchObservedRunningTime="2026-10-30 19:46:04.231481606 +0000 UTC m=+2395.972472685"
Oct 30 19:46:04.330000 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:04.329943    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"]
Oct 30 19:46:04.362190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:04.361670    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq" podUID="db5e3658-2a84-43ee-8390-e8f88ae56ea0" containerName="istio-proxy" containerID="cri-o://25426ac8e6379ca5da5c5198f7583be3867fde176eef78de076032ac5349154c" gracePeriod=30
Oct 30 19:46:04.523274 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:04.474823    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused" start-of-body=
Oct 30 19:46:04.523274 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:04.474884    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79" podUID="5c074845-90e7-46eb-bed0-9360969cf36e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused"
Oct 30 19:46:04.625456 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:04.625418    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:04.696157 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:04.690623    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:05.242307 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:05.242273    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:05.243411 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:05.243383    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:46:05.426367 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:05.426330    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused" start-of-body=
Oct 30 19:46:05.426556 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:05.426390    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79" podUID="5c074845-90e7-46eb-bed0-9360969cf36e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.182:15021/healthz/ready\": dial tcp 10.133.0.182:15021: connect: connection refused"
Oct 30 19:46:06.431748 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:06.431718    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:46:06.432224 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:06.432203    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:46:06.432908 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:06.432893    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:46:09.661381 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.661324    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:46:09.746812 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.746772    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-credential-socket\") pod \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") "
Oct 30 19:46:09.746999 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.746840    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-workload-socket\") pod \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") "
Oct 30 19:46:09.746999 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.746900    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-workload-certs\") pod \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") "
Oct 30 19:46:09.746999 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.746964    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-data\") pod \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") "
Oct 30 19:46:09.747200 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.747080    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istiod-ca-cert\") pod \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") "
Oct 30 19:46:09.747200 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.747084    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-credential-socket" pod "db5e3658-2a84-43ee-8390-e8f88ae56ea0" (UID: "db5e3658-2a84-43ee-8390-e8f88ae56ea0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:09.747200 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.747126    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-token\" (UniqueName: \"kubernetes.io/projected/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-token\") pod \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") "
Oct 30 19:46:09.747373 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.747267    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-data" pod "db5e3658-2a84-43ee-8390-e8f88ae56ea0" (UID: "db5e3658-2a84-43ee-8390-e8f88ae56ea0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:09.747373 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.747345    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-envoy\") pod \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") "
Oct 30 19:46:09.747492 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.747430    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db5e3658-2a84-43ee-8390-e8f88ae56ea0-kube-api-access-jvcbf\" (UniqueName: \"kubernetes.io/projected/db5e3658-2a84-43ee-8390-e8f88ae56ea0-kube-api-access-jvcbf\") pod \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") "
Oct 30 19:46:09.747492 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.747472    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-podinfo\") pod \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\" (UID: \"db5e3658-2a84-43ee-8390-e8f88ae56ea0\") "
Oct 30 19:46:09.747603 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.747515    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istiod-ca-cert" pod "db5e3658-2a84-43ee-8390-e8f88ae56ea0" (UID: "db5e3658-2a84-43ee-8390-e8f88ae56ea0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:09.747900 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.747821    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-workload-socket" pod "db5e3658-2a84-43ee-8390-e8f88ae56ea0" (UID: "db5e3658-2a84-43ee-8390-e8f88ae56ea0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:09.747900 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.747862    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-workload-certs" pod "db5e3658-2a84-43ee-8390-e8f88ae56ea0" (UID: "db5e3658-2a84-43ee-8390-e8f88ae56ea0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:09.749472 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.749446    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:09.749570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.749479    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:09.749570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.749494    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:09.749570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.749507    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:09.749570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.749521    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:09.756946 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.756909    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-podinfo" pod "db5e3658-2a84-43ee-8390-e8f88ae56ea0" (UID: "db5e3658-2a84-43ee-8390-e8f88ae56ea0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:46:09.757474 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.757440    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-token" pod "db5e3658-2a84-43ee-8390-e8f88ae56ea0" (UID: "db5e3658-2a84-43ee-8390-e8f88ae56ea0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:09.757637 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.757611    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db5e3658-2a84-43ee-8390-e8f88ae56ea0-kube-api-access-jvcbf" pod "db5e3658-2a84-43ee-8390-e8f88ae56ea0" (UID: "db5e3658-2a84-43ee-8390-e8f88ae56ea0"). InnerVolumeSpecName "kube-api-access-jvcbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:09.758514 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.758489    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-envoy" pod "db5e3658-2a84-43ee-8390-e8f88ae56ea0" (UID: "db5e3658-2a84-43ee-8390-e8f88ae56ea0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:09.851041 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.851002    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:09.851041 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.851035    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:09.851275 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.851049    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvcbf\" (UniqueName: \"kubernetes.io/projected/db5e3658-2a84-43ee-8390-e8f88ae56ea0-kube-api-access-jvcbf\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:09.851275 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:09.851063    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db5e3658-2a84-43ee-8390-e8f88ae56ea0-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:10.195688 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.195657    4524 scope.go:122] "RemoveContainer" containerID="7f8b518634a403388e1659c3b800eec20b4f544425c988518b164c095a851a4b"
Oct 30 19:46:10.209513 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.209479    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk" podUID="533f548c-cf0b-4bd7-9fed-2d9bde66ec95" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:10.223443 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.223417    4524 scope.go:122] "RemoveContainer" containerID="684f1dde6618fa1403bfd02c563a31d343c0b53805db59de4e7719241b53c13d"
Oct 30 19:46:10.459242 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.449307    4524 scope.go:122] "RemoveContainer" containerID="25426ac8e6379ca5da5c5198f7583be3867fde176eef78de076032ac5349154c"
Oct 30 19:46:10.543868 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.538529    4524 generic.go:356] "Generic (PLEG): container finished" podID="db5e3658-2a84-43ee-8390-e8f88ae56ea0" containerID="25426ac8e6379ca5da5c5198f7583be3867fde176eef78de076032ac5349154c" exitCode=0
Oct 30 19:46:10.543868 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.538774    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"
Oct 30 19:46:10.568002 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.566821    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq" event={"ID":"db5e3658-2a84-43ee-8390-e8f88ae56ea0","Type":"ContainerDied","Data":"25426ac8e6379ca5da5c5198f7583be3867fde176eef78de076032ac5349154c"}
Oct 30 19:46:10.568002 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.566904    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq" event={"ID":"db5e3658-2a84-43ee-8390-e8f88ae56ea0","Type":"ContainerDied","Data":"3e5a36a9100f8e5aaca42d075cd1882940f7890f13b45ae19145eb242ec75a1b"}
Oct 30 19:46:10.568002 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.566948    4524 scope.go:122] "RemoveContainer" containerID="25426ac8e6379ca5da5c5198f7583be3867fde176eef78de076032ac5349154c"
Oct 30 19:46:10.580351 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:46:10.580317    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25426ac8e6379ca5da5c5198f7583be3867fde176eef78de076032ac5349154c\": container with ID starting with 25426ac8e6379ca5da5c5198f7583be3867fde176eef78de076032ac5349154c not found: ID does not exist" containerID="25426ac8e6379ca5da5c5198f7583be3867fde176eef78de076032ac5349154c"
Oct 30 19:46:10.580474 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.580360    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25426ac8e6379ca5da5c5198f7583be3867fde176eef78de076032ac5349154c"} err="failed to get container status \"25426ac8e6379ca5da5c5198f7583be3867fde176eef78de076032ac5349154c\": rpc error: code = NotFound desc = could not find container \"25426ac8e6379ca5da5c5198f7583be3867fde176eef78de076032ac5349154c\": container with ID starting with 25426ac8e6379ca5da5c5198f7583be3867fde176eef78de076032ac5349154c not found: ID does not exist"
Oct 30 19:46:10.580474 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.580386    4524 scope.go:122] "RemoveContainer" containerID="25426ac8e6379ca5da5c5198f7583be3867fde176eef78de076032ac5349154c"
Oct 30 19:46:10.618667 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.587460    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25426ac8e6379ca5da5c5198f7583be3867fde176eef78de076032ac5349154c"} err="failed to get container status \"25426ac8e6379ca5da5c5198f7583be3867fde176eef78de076032ac5349154c\": rpc error: code = NotFound desc = could not find container \"25426ac8e6379ca5da5c5198f7583be3867fde176eef78de076032ac5349154c\": container with ID starting with 25426ac8e6379ca5da5c5198f7583be3867fde176eef78de076032ac5349154c not found: ID does not exist"
Oct 30 19:46:10.669203 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.663226    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"]
Oct 30 19:46:10.669853 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.669271    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-e3586f-openshift-default-976787cfc-v5brq"]
Oct 30 19:46:10.837191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.830837    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"]
Oct 30 19:46:10.837191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.832072    4524 cpu_manager.go:414] "containerMap: removing container" podUID="db5e3658-2a84-43ee-8390-e8f88ae56ea0" containerName="istio-proxy"
Oct 30 19:46:10.837191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.832091    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db5e3658-2a84-43ee-8390-e8f88ae56ea0" containerName="istio-proxy"
Oct 30 19:46:10.837191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.832347    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="db5e3658-2a84-43ee-8390-e8f88ae56ea0" containerName="istio-proxy"
Oct 30 19:46:10.842194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.840270    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"
Oct 30 19:46:10.846191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.845924    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-aa49c5-serving-cert\""
Oct 30 19:46:10.879270 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.879241    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"]
Oct 30 19:46:10.997266 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.997187    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/944c40c9-95ff-419a-8429-2e8c9a2eefba-metrics-certs\") pod \"maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7\" (UID: \"944c40c9-95ff-419a-8429-2e8c9a2eefba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"
Oct 30 19:46:10.997442 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.997424    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw6kq\" (UniqueName: \"kubernetes.io/projected/944c40c9-95ff-419a-8429-2e8c9a2eefba-kube-api-access-bw6kq\") pod \"maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7\" (UID: \"944c40c9-95ff-419a-8429-2e8c9a2eefba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"
Oct 30 19:46:10.997569 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.997552    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/944c40c9-95ff-419a-8429-2e8c9a2eefba-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7\" (UID: \"944c40c9-95ff-419a-8429-2e8c9a2eefba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"
Oct 30 19:46:10.997619 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:10.997578    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/944c40c9-95ff-419a-8429-2e8c9a2eefba-maas-api-tls\") pod \"maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7\" (UID: \"944c40c9-95ff-419a-8429-2e8c9a2eefba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"
Oct 30 19:46:11.109771 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:11.099158    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/944c40c9-95ff-419a-8429-2e8c9a2eefba-metrics-certs\") pod \"maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7\" (UID: \"944c40c9-95ff-419a-8429-2e8c9a2eefba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"
Oct 30 19:46:11.109771 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:11.109064    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw6kq\" (UniqueName: \"kubernetes.io/projected/944c40c9-95ff-419a-8429-2e8c9a2eefba-kube-api-access-bw6kq\") pod \"maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7\" (UID: \"944c40c9-95ff-419a-8429-2e8c9a2eefba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"
Oct 30 19:46:11.109771 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:11.109293    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/944c40c9-95ff-419a-8429-2e8c9a2eefba-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7\" (UID: \"944c40c9-95ff-419a-8429-2e8c9a2eefba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"
Oct 30 19:46:11.109771 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:11.109330    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/944c40c9-95ff-419a-8429-2e8c9a2eefba-maas-api-tls\") pod \"maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7\" (UID: \"944c40c9-95ff-419a-8429-2e8c9a2eefba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"
Oct 30 19:46:11.114002 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:11.111538    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/944c40c9-95ff-419a-8429-2e8c9a2eefba-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7\" (UID: \"944c40c9-95ff-419a-8429-2e8c9a2eefba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"
Oct 30 19:46:11.114002 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:11.113086    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/944c40c9-95ff-419a-8429-2e8c9a2eefba-maas-api-tls\") pod \"maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7\" (UID: \"944c40c9-95ff-419a-8429-2e8c9a2eefba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"
Oct 30 19:46:11.118054 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:11.114965    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/944c40c9-95ff-419a-8429-2e8c9a2eefba-metrics-certs\") pod \"maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7\" (UID: \"944c40c9-95ff-419a-8429-2e8c9a2eefba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"
Oct 30 19:46:11.137770 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:11.130577    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw6kq\" (UniqueName: \"kubernetes.io/projected/944c40c9-95ff-419a-8429-2e8c9a2eefba-kube-api-access-bw6kq\") pod \"maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7\" (UID: \"944c40c9-95ff-419a-8429-2e8c9a2eefba\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"
Oct 30 19:46:11.167352 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:11.167325    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"
Oct 30 19:46:11.236744 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:11.236629    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87" podUID="1c50188a-2ca9-4063-9092-9fdd095582aa" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:11.572190 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:11.562753    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"]
Oct 30 19:46:11.647989 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:11.647931    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"]
Oct 30 19:46:11.648209 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:11.648157    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b" podUID="0183f05e-6c5a-4614-ac22-d050be53409f" containerName="maas-api" containerID="cri-o://43a0455957e3dae934eeb83e0a97213c64cdbcb55d11367c5bc9ff00220ad589" gracePeriod=30
Oct 30 19:46:11.934075 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:11.934046    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"
Oct 30 19:46:12.080193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.079701    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0183f05e-6c5a-4614-ac22-d050be53409f-kube-api-access-zmq78\" (UniqueName: \"kubernetes.io/projected/0183f05e-6c5a-4614-ac22-d050be53409f-kube-api-access-zmq78\") pod \"0183f05e-6c5a-4614-ac22-d050be53409f\" (UID: \"0183f05e-6c5a-4614-ac22-d050be53409f\") "
Oct 30 19:46:12.080193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.079822    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0183f05e-6c5a-4614-ac22-d050be53409f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0183f05e-6c5a-4614-ac22-d050be53409f-metrics-certs\") pod \"0183f05e-6c5a-4614-ac22-d050be53409f\" (UID: \"0183f05e-6c5a-4614-ac22-d050be53409f\") "
Oct 30 19:46:12.080193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.079861    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0183f05e-6c5a-4614-ac22-d050be53409f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0183f05e-6c5a-4614-ac22-d050be53409f-maas-api-tls\") pod \"0183f05e-6c5a-4614-ac22-d050be53409f\" (UID: \"0183f05e-6c5a-4614-ac22-d050be53409f\") "
Oct 30 19:46:12.080193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.079941    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0183f05e-6c5a-4614-ac22-d050be53409f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0183f05e-6c5a-4614-ac22-d050be53409f-openshift-service-ca\") pod \"0183f05e-6c5a-4614-ac22-d050be53409f\" (UID: \"0183f05e-6c5a-4614-ac22-d050be53409f\") "
Oct 30 19:46:12.196252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.117397    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0183f05e-6c5a-4614-ac22-d050be53409f-openshift-service-ca" pod "0183f05e-6c5a-4614-ac22-d050be53409f" (UID: "0183f05e-6c5a-4614-ac22-d050be53409f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:12.196252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.185495    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0183f05e-6c5a-4614-ac22-d050be53409f-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:12.302969 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.301347    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0183f05e-6c5a-4614-ac22-d050be53409f-metrics-certs" pod "0183f05e-6c5a-4614-ac22-d050be53409f" (UID: "0183f05e-6c5a-4614-ac22-d050be53409f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:12.302969 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.301379    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0183f05e-6c5a-4614-ac22-d050be53409f-kube-api-access-zmq78" pod "0183f05e-6c5a-4614-ac22-d050be53409f" (UID: "0183f05e-6c5a-4614-ac22-d050be53409f"). InnerVolumeSpecName "kube-api-access-zmq78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:12.302969 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.302827    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmq78\" (UniqueName: \"kubernetes.io/projected/0183f05e-6c5a-4614-ac22-d050be53409f-kube-api-access-zmq78\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:12.302969 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.302855    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0183f05e-6c5a-4614-ac22-d050be53409f-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:12.305263 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.305238    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0183f05e-6c5a-4614-ac22-d050be53409f-maas-api-tls" pod "0183f05e-6c5a-4614-ac22-d050be53409f" (UID: "0183f05e-6c5a-4614-ac22-d050be53409f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:12.392033 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.391998    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db5e3658-2a84-43ee-8390-e8f88ae56ea0" path="/var/lib/kubelet/pods/db5e3658-2a84-43ee-8390-e8f88ae56ea0/volumes"
Oct 30 19:46:12.403962 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.403939    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0183f05e-6c5a-4614-ac22-d050be53409f-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:12.615297 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.615263    4524 generic.go:356] "Generic (PLEG): container finished" podID="0183f05e-6c5a-4614-ac22-d050be53409f" containerID="43a0455957e3dae934eeb83e0a97213c64cdbcb55d11367c5bc9ff00220ad589" exitCode=0
Oct 30 19:46:12.615492 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.615317    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"
Oct 30 19:46:12.615492 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.615337    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b" event={"ID":"0183f05e-6c5a-4614-ac22-d050be53409f","Type":"ContainerDied","Data":"43a0455957e3dae934eeb83e0a97213c64cdbcb55d11367c5bc9ff00220ad589"}
Oct 30 19:46:12.615492 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.615404    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b" event={"ID":"0183f05e-6c5a-4614-ac22-d050be53409f","Type":"ContainerDied","Data":"26d9b7b5cb1316a2dd118bbe8adda80485854e2ba04249862a6d84aac886c499"}
Oct 30 19:46:12.615492 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.615435    4524 scope.go:122] "RemoveContainer" containerID="43a0455957e3dae934eeb83e0a97213c64cdbcb55d11367c5bc9ff00220ad589"
Oct 30 19:46:12.619544 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.619504    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7" event={"ID":"944c40c9-95ff-419a-8429-2e8c9a2eefba","Type":"ContainerStarted","Data":"5cfe27a6aabaaf14814af578b77b34e90f4c7a5a0884e179eba1ed6997455bf5"}
Oct 30 19:46:12.619684 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.619557    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7" event={"ID":"944c40c9-95ff-419a-8429-2e8c9a2eefba","Type":"ContainerStarted","Data":"6cc4b5efe829ddc10d085c24c0f9e0a4a854813ed965c65789025707e04ab169"}
Oct 30 19:46:12.619786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.619755    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"
Oct 30 19:46:12.628503 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.628484    4524 scope.go:122] "RemoveContainer" containerID="43a0455957e3dae934eeb83e0a97213c64cdbcb55d11367c5bc9ff00220ad589"
Oct 30 19:46:12.628807 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:46:12.628785    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43a0455957e3dae934eeb83e0a97213c64cdbcb55d11367c5bc9ff00220ad589\": container with ID starting with 43a0455957e3dae934eeb83e0a97213c64cdbcb55d11367c5bc9ff00220ad589 not found: ID does not exist" containerID="43a0455957e3dae934eeb83e0a97213c64cdbcb55d11367c5bc9ff00220ad589"
Oct 30 19:46:12.628861 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.628818    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43a0455957e3dae934eeb83e0a97213c64cdbcb55d11367c5bc9ff00220ad589"} err="failed to get container status \"43a0455957e3dae934eeb83e0a97213c64cdbcb55d11367c5bc9ff00220ad589\": rpc error: code = NotFound desc = could not find container \"43a0455957e3dae934eeb83e0a97213c64cdbcb55d11367c5bc9ff00220ad589\": container with ID starting with 43a0455957e3dae934eeb83e0a97213c64cdbcb55d11367c5bc9ff00220ad589 not found: ID does not exist"
Oct 30 19:46:12.636829 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.636784    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"]
Oct 30 19:46:12.643453 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.643427    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-2405fb-76c8bd66bd-wt24b"]
Oct 30 19:46:12.658760 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:12.658715    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7" podStartSLOduration=1.9337070170000001 podStartE2EDuration="2.658702872s" totalImagesPullingTime="724.995855ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:46:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:46:12.657463428 +0000 UTC m=+2404.398454509" watchObservedRunningTime="2026-10-30 19:46:12.658702872 +0000 UTC m=+2404.399693951"
Oct 30 19:46:14.396585 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:14.396556    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0183f05e-6c5a-4614-ac22-d050be53409f" path="/var/lib/kubelet/pods/0183f05e-6c5a-4614-ac22-d050be53409f/volumes"
Oct 30 19:46:15.112300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:15.112263    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5"]
Oct 30 19:46:15.142294 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:15.133602    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d"]
Oct 30 19:46:15.646434 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:46:15.646396    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:46:18.634120 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:18.634093    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"
Oct 30 19:46:19.746450 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:19.746391    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"]
Oct 30 19:46:19.751574 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:19.751426    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv" podUID="0ad9005d-b179-48d1-8a9f-67fff6724a6f" containerName="istio-proxy" containerID="cri-o://ff4d2730b5a342a1814c559e5c7442f882394ca42b8825dada709b95477ca051" gracePeriod=30
Oct 30 19:46:19.755477 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:19.755449    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"]
Oct 30 19:46:19.755687 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:19.755662    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj" podUID="f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" containerName="istio-proxy" containerID="cri-o://1f13892b8c237be127bbc24d152df5bd364541da9700573024572b9770672eb5" gracePeriod=30
Oct 30 19:46:20.129232 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.121003    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5" podUID="5355e2dd-ba58-4602-bedc-44a2979762e1" containerName="payload-processing" containerID="cri-o://77e7d5836186b50b3e4ebbe4a881cb0b05e5d69511a784fd468220fee8ef5e59" gracePeriod=25
Oct 30 19:46:20.134323 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.134279    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d" podUID="34a1d722-a8a4-45e0-a5e4-6306c70eb7bb" containerName="payload-pre-processing" containerID="cri-o://617aca740b732e2809707bb638e6595aba570469fbc6e61a7ea61ca149679b2f" gracePeriod=25
Oct 30 19:46:20.210558 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:46:20.210526    4524 logging.go:55] [core] [Channel #7371 SubChannel #7372]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.106:50052", ServerName: "10.133.0.106:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.106:50052: connect: connection refused"
Oct 30 19:46:20.431646 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.431622    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk"
Oct 30 19:46:20.435548 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.435527    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87"
Oct 30 19:46:20.446275 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.446248    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/533f548c-cf0b-4bd7-9fed-2d9bde66ec95-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/533f548c-cf0b-4bd7-9fed-2d9bde66ec95-plugins-config-volume\") pod \"533f548c-cf0b-4bd7-9fed-2d9bde66ec95\" (UID: \"533f548c-cf0b-4bd7-9fed-2d9bde66ec95\") "
Oct 30 19:46:20.446618 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.446594    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/533f548c-cf0b-4bd7-9fed-2d9bde66ec95-plugins-config-volume" pod "533f548c-cf0b-4bd7-9fed-2d9bde66ec95" (UID: "533f548c-cf0b-4bd7-9fed-2d9bde66ec95"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:20.446837 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.446819    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/533f548c-cf0b-4bd7-9fed-2d9bde66ec95-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:20.547458 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.547426    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1c50188a-2ca9-4063-9092-9fdd095582aa-kube-api-access-dgc4p\" (UniqueName: \"kubernetes.io/projected/1c50188a-2ca9-4063-9092-9fdd095582aa-kube-api-access-dgc4p\") pod \"1c50188a-2ca9-4063-9092-9fdd095582aa\" (UID: \"1c50188a-2ca9-4063-9092-9fdd095582aa\") "
Oct 30 19:46:20.547661 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.547541    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/533f548c-cf0b-4bd7-9fed-2d9bde66ec95-kube-api-access-pnrtx\" (UniqueName: \"kubernetes.io/projected/533f548c-cf0b-4bd7-9fed-2d9bde66ec95-kube-api-access-pnrtx\") pod \"533f548c-cf0b-4bd7-9fed-2d9bde66ec95\" (UID: \"533f548c-cf0b-4bd7-9fed-2d9bde66ec95\") "
Oct 30 19:46:20.547661 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.547568    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1c50188a-2ca9-4063-9092-9fdd095582aa-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c50188a-2ca9-4063-9092-9fdd095582aa-plugins-config-volume\") pod \"1c50188a-2ca9-4063-9092-9fdd095582aa\" (UID: \"1c50188a-2ca9-4063-9092-9fdd095582aa\") "
Oct 30 19:46:20.548303 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.548270    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c50188a-2ca9-4063-9092-9fdd095582aa-plugins-config-volume" pod "1c50188a-2ca9-4063-9092-9fdd095582aa" (UID: "1c50188a-2ca9-4063-9092-9fdd095582aa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:20.549818 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.549793    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c50188a-2ca9-4063-9092-9fdd095582aa-kube-api-access-dgc4p" pod "1c50188a-2ca9-4063-9092-9fdd095582aa" (UID: "1c50188a-2ca9-4063-9092-9fdd095582aa"). InnerVolumeSpecName "kube-api-access-dgc4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:20.550237 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.550209    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/533f548c-cf0b-4bd7-9fed-2d9bde66ec95-kube-api-access-pnrtx" pod "533f548c-cf0b-4bd7-9fed-2d9bde66ec95" (UID: "533f548c-cf0b-4bd7-9fed-2d9bde66ec95"). InnerVolumeSpecName "kube-api-access-pnrtx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:20.649596 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.649520    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgc4p\" (UniqueName: \"kubernetes.io/projected/1c50188a-2ca9-4063-9092-9fdd095582aa-kube-api-access-dgc4p\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:20.649596 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.649549    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnrtx\" (UniqueName: \"kubernetes.io/projected/533f548c-cf0b-4bd7-9fed-2d9bde66ec95-kube-api-access-pnrtx\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:20.649596 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.649558    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1c50188a-2ca9-4063-9092-9fdd095582aa-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:20.754705 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.754671    4524 generic.go:356] "Generic (PLEG): container finished" podID="533f548c-cf0b-4bd7-9fed-2d9bde66ec95" containerID="82e3e1defc32af005ce07a40e50bc0289314db1694f99cc9924d6ed98b6099d0" exitCode=0
Oct 30 19:46:20.755209 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.754726    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk"
Oct 30 19:46:20.755209 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.754755    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk" event={"ID":"533f548c-cf0b-4bd7-9fed-2d9bde66ec95","Type":"ContainerDied","Data":"82e3e1defc32af005ce07a40e50bc0289314db1694f99cc9924d6ed98b6099d0"}
Oct 30 19:46:20.755209 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.754805    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk" event={"ID":"533f548c-cf0b-4bd7-9fed-2d9bde66ec95","Type":"ContainerDied","Data":"7e05cefd03ac89025081b9fa745b400b8646105c0f9136d1439647a1b78a5d41"}
Oct 30 19:46:20.755209 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.754832    4524 scope.go:122] "RemoveContainer" containerID="82e3e1defc32af005ce07a40e50bc0289314db1694f99cc9924d6ed98b6099d0"
Oct 30 19:46:20.756342 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.756320    4524 generic.go:356] "Generic (PLEG): container finished" podID="1c50188a-2ca9-4063-9092-9fdd095582aa" containerID="5fe9601fadd8cf475b64efba769a18e4dac62beaa7e097d3cb983176d0c33000" exitCode=0
Oct 30 19:46:20.756460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.756358    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87"
Oct 30 19:46:20.756460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.756418    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87" event={"ID":"1c50188a-2ca9-4063-9092-9fdd095582aa","Type":"ContainerDied","Data":"5fe9601fadd8cf475b64efba769a18e4dac62beaa7e097d3cb983176d0c33000"}
Oct 30 19:46:20.756586 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.756480    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87" event={"ID":"1c50188a-2ca9-4063-9092-9fdd095582aa","Type":"ContainerDied","Data":"711ce73886786c62cd4d5b87c40028689d33875d681c71800a8f19825b22d253"}
Oct 30 19:46:20.767210 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.767116    4524 scope.go:122] "RemoveContainer" containerID="82e3e1defc32af005ce07a40e50bc0289314db1694f99cc9924d6ed98b6099d0"
Oct 30 19:46:20.767480 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:46:20.767458    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82e3e1defc32af005ce07a40e50bc0289314db1694f99cc9924d6ed98b6099d0\": container with ID starting with 82e3e1defc32af005ce07a40e50bc0289314db1694f99cc9924d6ed98b6099d0 not found: ID does not exist" containerID="82e3e1defc32af005ce07a40e50bc0289314db1694f99cc9924d6ed98b6099d0"
Oct 30 19:46:20.767613 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.767586    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82e3e1defc32af005ce07a40e50bc0289314db1694f99cc9924d6ed98b6099d0"} err="failed to get container status \"82e3e1defc32af005ce07a40e50bc0289314db1694f99cc9924d6ed98b6099d0\": rpc error: code = NotFound desc = could not find container \"82e3e1defc32af005ce07a40e50bc0289314db1694f99cc9924d6ed98b6099d0\": container with ID starting with 82e3e1defc32af005ce07a40e50bc0289314db1694f99cc9924d6ed98b6099d0 not found: ID does not exist"
Oct 30 19:46:20.767613 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.767614    4524 scope.go:122] "RemoveContainer" containerID="5fe9601fadd8cf475b64efba769a18e4dac62beaa7e097d3cb983176d0c33000"
Oct 30 19:46:20.778598 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.778578    4524 scope.go:122] "RemoveContainer" containerID="5fe9601fadd8cf475b64efba769a18e4dac62beaa7e097d3cb983176d0c33000"
Oct 30 19:46:20.786113 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.786083    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk"]
Oct 30 19:46:20.790485 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:46:20.790462    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fe9601fadd8cf475b64efba769a18e4dac62beaa7e097d3cb983176d0c33000\": container with ID starting with 5fe9601fadd8cf475b64efba769a18e4dac62beaa7e097d3cb983176d0c33000 not found: ID does not exist" containerID="5fe9601fadd8cf475b64efba769a18e4dac62beaa7e097d3cb983176d0c33000"
Oct 30 19:46:20.790565 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.790490    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fe9601fadd8cf475b64efba769a18e4dac62beaa7e097d3cb983176d0c33000"} err="failed to get container status \"5fe9601fadd8cf475b64efba769a18e4dac62beaa7e097d3cb983176d0c33000\": rpc error: code = NotFound desc = could not find container \"5fe9601fadd8cf475b64efba769a18e4dac62beaa7e097d3cb983176d0c33000\": container with ID starting with 5fe9601fadd8cf475b64efba769a18e4dac62beaa7e097d3cb983176d0c33000 not found: ID does not exist"
Oct 30 19:46:20.793619 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.793578    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk"]
Oct 30 19:46:20.800066 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.800046    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87"]
Oct 30 19:46:20.806109 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.806088    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-2405fb-9f8fbdf8-4mq87"]
Oct 30 19:46:20.899940 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.899866    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:46:20.899940 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:20.899925    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj" podUID="f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:46:21.210091 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:21.210006    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w0-2405fb-7b444456dc-b27kk" podUID="533f548c-cf0b-4bd7-9fed-2d9bde66ec95" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.106:50052\" within 1s: context deadline exceeded"
Oct 30 19:46:22.396360 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:22.396323    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1c50188a-2ca9-4063-9092-9fdd095582aa" path="/var/lib/kubelet/pods/1c50188a-2ca9-4063-9092-9fdd095582aa/volumes"
Oct 30 19:46:22.396923 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:22.396903    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="533f548c-cf0b-4bd7-9fed-2d9bde66ec95" path="/var/lib/kubelet/pods/533f548c-cf0b-4bd7-9fed-2d9bde66ec95/volumes"
Oct 30 19:46:22.504404 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:22.504356    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5" podUID="5355e2dd-ba58-4602-bedc-44a2979762e1" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:23.423692 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:23.423643    4524 patch_prober.go:29] interesting pod/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:46:23.424242 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:23.423712    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv" podUID="0ad9005d-b179-48d1-8a9f-67fff6724a6f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:46:23.510398 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:23.510355    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d" podUID="34a1d722-a8a4-45e0-a5e4-6306c70eb7bb" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:24.737297 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:24.737259    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"]
Oct 30 19:46:24.741953 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:24.741924    4524 cpu_manager.go:414] "containerMap: removing container" podUID="1c50188a-2ca9-4063-9092-9fdd095582aa" containerName="payload-processing"
Oct 30 19:46:24.741953 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:24.741954    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1c50188a-2ca9-4063-9092-9fdd095582aa" containerName="payload-processing"
Oct 30 19:46:24.742242 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:24.741995    4524 cpu_manager.go:414] "containerMap: removing container" podUID="533f548c-cf0b-4bd7-9fed-2d9bde66ec95" containerName="payload-pre-processing"
Oct 30 19:46:24.742242 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:24.742005    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="533f548c-cf0b-4bd7-9fed-2d9bde66ec95" containerName="payload-pre-processing"
Oct 30 19:46:24.742242 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:24.742016    4524 cpu_manager.go:414] "containerMap: removing container" podUID="0183f05e-6c5a-4614-ac22-d050be53409f" containerName="maas-api"
Oct 30 19:46:24.742242 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:24.742024    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0183f05e-6c5a-4614-ac22-d050be53409f" containerName="maas-api"
Oct 30 19:46:24.742242 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:24.742239    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1c50188a-2ca9-4063-9092-9fdd095582aa" containerName="payload-processing"
Oct 30 19:46:24.742631 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:24.742257    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="533f548c-cf0b-4bd7-9fed-2d9bde66ec95" containerName="payload-pre-processing"
Oct 30 19:46:24.742631 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:24.742266    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="0183f05e-6c5a-4614-ac22-d050be53409f" containerName="maas-api"
Oct 30 19:46:24.752656 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:24.752634    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"
Oct 30 19:46:24.759379 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:24.759352    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"]
Oct 30 19:46:24.759561 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:24.759460    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-c0a27e-serving-cert\""
Oct 30 19:46:24.916205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:24.909248    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eadb3435-17d0-4041-8210-3b5cfe7d50bc-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k\" (UID: \"eadb3435-17d0-4041-8210-3b5cfe7d50bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"
Oct 30 19:46:24.916205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:24.914033    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fkkw\" (UniqueName: \"kubernetes.io/projected/eadb3435-17d0-4041-8210-3b5cfe7d50bc-kube-api-access-6fkkw\") pod \"maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k\" (UID: \"eadb3435-17d0-4041-8210-3b5cfe7d50bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"
Oct 30 19:46:24.916205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:24.914155    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eadb3435-17d0-4041-8210-3b5cfe7d50bc-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k\" (UID: \"eadb3435-17d0-4041-8210-3b5cfe7d50bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"
Oct 30 19:46:24.916205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:24.914427    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eadb3435-17d0-4041-8210-3b5cfe7d50bc-metrics-certs\") pod \"maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k\" (UID: \"eadb3435-17d0-4041-8210-3b5cfe7d50bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"
Oct 30 19:46:25.016462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.016365    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eadb3435-17d0-4041-8210-3b5cfe7d50bc-metrics-certs\") pod \"maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k\" (UID: \"eadb3435-17d0-4041-8210-3b5cfe7d50bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"
Oct 30 19:46:25.016462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.016413    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eadb3435-17d0-4041-8210-3b5cfe7d50bc-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k\" (UID: \"eadb3435-17d0-4041-8210-3b5cfe7d50bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"
Oct 30 19:46:25.016742 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.016546    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fkkw\" (UniqueName: \"kubernetes.io/projected/eadb3435-17d0-4041-8210-3b5cfe7d50bc-kube-api-access-6fkkw\") pod \"maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k\" (UID: \"eadb3435-17d0-4041-8210-3b5cfe7d50bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"
Oct 30 19:46:25.016742 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.016592    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eadb3435-17d0-4041-8210-3b5cfe7d50bc-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k\" (UID: \"eadb3435-17d0-4041-8210-3b5cfe7d50bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"
Oct 30 19:46:25.017289 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.017259    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eadb3435-17d0-4041-8210-3b5cfe7d50bc-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k\" (UID: \"eadb3435-17d0-4041-8210-3b5cfe7d50bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"
Oct 30 19:46:25.020066 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.020039    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eadb3435-17d0-4041-8210-3b5cfe7d50bc-metrics-certs\") pod \"maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k\" (UID: \"eadb3435-17d0-4041-8210-3b5cfe7d50bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"
Oct 30 19:46:25.023755 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.023731    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eadb3435-17d0-4041-8210-3b5cfe7d50bc-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k\" (UID: \"eadb3435-17d0-4041-8210-3b5cfe7d50bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"
Oct 30 19:46:25.029871 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.029847    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fkkw\" (UniqueName: \"kubernetes.io/projected/eadb3435-17d0-4041-8210-3b5cfe7d50bc-kube-api-access-6fkkw\") pod \"maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k\" (UID: \"eadb3435-17d0-4041-8210-3b5cfe7d50bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"
Oct 30 19:46:25.080656 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.080629    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"
Oct 30 19:46:25.173926 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.173573    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:46:25.279699 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.279633    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:46:25.346705 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.344085    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-kube-api-access-5gpnh\" (UniqueName: \"kubernetes.io/projected/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-kube-api-access-5gpnh\") pod \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") "
Oct 30 19:46:25.346705 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.344234    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-data\") pod \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") "
Oct 30 19:46:25.346705 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.344288    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-workload-socket\") pod \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") "
Oct 30 19:46:25.346705 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.344335    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-podinfo\") pod \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") "
Oct 30 19:46:25.346705 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.344379    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-credential-socket\") pod \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") "
Oct 30 19:46:25.346705 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.344440    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-envoy\") pod \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") "
Oct 30 19:46:25.346705 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.344507    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istiod-ca-cert\") pod \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") "
Oct 30 19:46:25.346705 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.344674    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-token\" (UniqueName: \"kubernetes.io/projected/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-token\") pod \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") "
Oct 30 19:46:25.346705 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.344727    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-workload-certs\") pod \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\" (UID: \"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f\") "
Oct 30 19:46:25.346705 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.345784    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-workload-certs" pod "f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" (UID: "f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:25.346705 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.346686    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-credential-socket" pod "f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" (UID: "f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:25.347491 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.346859    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-data" pod "f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" (UID: "f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:25.347491 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.347077    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-workload-socket" pod "f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" (UID: "f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:25.348249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.348217    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istiod-ca-cert" pod "f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" (UID: "f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:25.370370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.366497    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-podinfo" pod "f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" (UID: "f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:46:25.370370 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.370310    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-token" pod "f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" (UID: "f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:25.379468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.379432    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-envoy" pod "f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" (UID: "f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:25.383369 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.383322    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-kube-api-access-5gpnh" pod "f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" (UID: "f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f"). InnerVolumeSpecName "kube-api-access-5gpnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:25.427233 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.423642    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"]
Oct 30 19:46:25.427233 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.423953    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc" podUID="a26c2511-7fe6-4a60-86f8-781c17fac337" containerName="maas-api" containerID="cri-o://ca99ebc94faea5f62cdb83ed0be289349a2db2368e4268c297f0d712c6ae3452" gracePeriod=30
Oct 30 19:46:25.447694 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.446878    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-workload-socket\") pod \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") "
Oct 30 19:46:25.447694 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.446973    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istiod-ca-cert\") pod \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") "
Oct 30 19:46:25.447694 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.447047    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-token\" (UniqueName: \"kubernetes.io/projected/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-token\") pod \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") "
Oct 30 19:46:25.447694 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.447157    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-envoy\") pod \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") "
Oct 30 19:46:25.447694 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.447241    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-data\") pod \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") "
Oct 30 19:46:25.447694 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.447278    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-workload-certs\") pod \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") "
Oct 30 19:46:25.447694 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.447380    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-podinfo\") pod \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") "
Oct 30 19:46:25.447694 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.447416    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0ad9005d-b179-48d1-8a9f-67fff6724a6f-kube-api-access-8htzp\" (UniqueName: \"kubernetes.io/projected/0ad9005d-b179-48d1-8a9f-67fff6724a6f-kube-api-access-8htzp\") pod \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") "
Oct 30 19:46:25.447694 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.447463    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-credential-socket\") pod \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\" (UID: \"0ad9005d-b179-48d1-8a9f-67fff6724a6f\") "
Oct 30 19:46:25.448345 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.448209    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"]
Oct 30 19:46:25.449402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.449031    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:25.449402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.449057    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:25.449402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.449072    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gpnh\" (UniqueName: \"kubernetes.io/projected/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-kube-api-access-5gpnh\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:25.449402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.449087    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:25.449402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.449102    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:25.449402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.449114    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:25.449402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.449127    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:25.449402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.449139    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:25.449402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.449152    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:25.449402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.449209    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-workload-socket" pod "0ad9005d-b179-48d1-8a9f-67fff6724a6f" (UID: "0ad9005d-b179-48d1-8a9f-67fff6724a6f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:25.449920 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.449591    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istiod-ca-cert" pod "0ad9005d-b179-48d1-8a9f-67fff6724a6f" (UID: "0ad9005d-b179-48d1-8a9f-67fff6724a6f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:25.449920 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.449707    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-data" pod "0ad9005d-b179-48d1-8a9f-67fff6724a6f" (UID: "0ad9005d-b179-48d1-8a9f-67fff6724a6f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:25.449920 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.449907    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-workload-certs" pod "0ad9005d-b179-48d1-8a9f-67fff6724a6f" (UID: "0ad9005d-b179-48d1-8a9f-67fff6724a6f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:25.451132 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.451101    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-credential-socket" pod "0ad9005d-b179-48d1-8a9f-67fff6724a6f" (UID: "0ad9005d-b179-48d1-8a9f-67fff6724a6f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:25.457212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.456318    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ad9005d-b179-48d1-8a9f-67fff6724a6f-kube-api-access-8htzp" pod "0ad9005d-b179-48d1-8a9f-67fff6724a6f" (UID: "0ad9005d-b179-48d1-8a9f-67fff6724a6f"). InnerVolumeSpecName "kube-api-access-8htzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:25.457212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.456956    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-podinfo" pod "0ad9005d-b179-48d1-8a9f-67fff6724a6f" (UID: "0ad9005d-b179-48d1-8a9f-67fff6724a6f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:46:25.462592 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.462556    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-envoy" pod "0ad9005d-b179-48d1-8a9f-67fff6724a6f" (UID: "0ad9005d-b179-48d1-8a9f-67fff6724a6f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:25.463599 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.463576    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-token" pod "0ad9005d-b179-48d1-8a9f-67fff6724a6f" (UID: "0ad9005d-b179-48d1-8a9f-67fff6724a6f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:25.486952 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:46:25.486915    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeadb3435_17d0_4041_8210_3b5cfe7d50bc.slice/crio-7a8d4ac39d73b91755ab2adbb1605fdd8ba26c29b31aad1082eb16eadc42d0ed WatchSource:0}: Error finding container 7a8d4ac39d73b91755ab2adbb1605fdd8ba26c29b31aad1082eb16eadc42d0ed: Status 404 returned error can't find the container with id 7a8d4ac39d73b91755ab2adbb1605fdd8ba26c29b31aad1082eb16eadc42d0ed
Oct 30 19:46:25.549690 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.549632    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2"]
Oct 30 19:46:25.550163 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.550132    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2" podUID="6ffe966b-ae60-467e-9c3d-6600bc846a6b" containerName="payload-pre-processing" containerID="cri-o://1227c1421eefe6f55aa3247de691c308ac184b0d46b02d46445b43796aefa76f" gracePeriod=30
Oct 30 19:46:25.561679 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.561650    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:25.561792 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.561684    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:25.561792 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.561697    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:25.561792 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.561709    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:25.561792 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.561722    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:25.561792 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.561736    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:25.561792 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.561748    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0ad9005d-b179-48d1-8a9f-67fff6724a6f-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:25.561792 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.561762    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8htzp\" (UniqueName: \"kubernetes.io/projected/0ad9005d-b179-48d1-8a9f-67fff6724a6f-kube-api-access-8htzp\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:25.561792 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.561777    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0ad9005d-b179-48d1-8a9f-67fff6724a6f-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:25.853332 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.853297    4524 generic.go:356] "Generic (PLEG): container finished" podID="f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" containerID="1f13892b8c237be127bbc24d152df5bd364541da9700573024572b9770672eb5" exitCode=0
Oct 30 19:46:25.853780 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.853441    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj" event={"ID":"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f","Type":"ContainerDied","Data":"1f13892b8c237be127bbc24d152df5bd364541da9700573024572b9770672eb5"}
Oct 30 19:46:25.853780 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.853499    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj" event={"ID":"f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f","Type":"ContainerDied","Data":"10624bba90ec243c78df544c67667efa61e6cbedcd0382a7e62edec5190e716e"}
Oct 30 19:46:25.853780 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.853529    4524 scope.go:122] "RemoveContainer" containerID="1f13892b8c237be127bbc24d152df5bd364541da9700573024572b9770672eb5"
Oct 30 19:46:25.853780 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.853720    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"
Oct 30 19:46:25.956299 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.955872    4524 generic.go:356] "Generic (PLEG): container finished" podID="0ad9005d-b179-48d1-8a9f-67fff6724a6f" containerID="ff4d2730b5a342a1814c559e5c7442f882394ca42b8825dada709b95477ca051" exitCode=0
Oct 30 19:46:25.973409 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.956556    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv" event={"ID":"0ad9005d-b179-48d1-8a9f-67fff6724a6f","Type":"ContainerDied","Data":"ff4d2730b5a342a1814c559e5c7442f882394ca42b8825dada709b95477ca051"}
Oct 30 19:46:25.978074 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.956712    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"
Oct 30 19:46:25.986796 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:25.973723    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv" event={"ID":"0ad9005d-b179-48d1-8a9f-67fff6724a6f","Type":"ContainerDied","Data":"ffaf98a9347e8d99ad0ebdbada4c9d87a92ff962a5ef2759fc57d0faa9ba4b22"}
Oct 30 19:46:26.022356 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.022315    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc" event={"ID":"a26c2511-7fe6-4a60-86f8-781c17fac337","Type":"ContainerDied","Data":"ca99ebc94faea5f62cdb83ed0be289349a2db2368e4268c297f0d712c6ae3452"}
Oct 30 19:46:26.024074 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.019310    4524 generic.go:356] "Generic (PLEG): container finished" podID="a26c2511-7fe6-4a60-86f8-781c17fac337" containerID="ca99ebc94faea5f62cdb83ed0be289349a2db2368e4268c297f0d712c6ae3452" exitCode=0
Oct 30 19:46:26.027101 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.026901    4524 generic.go:356] "Generic (PLEG): container finished" podID="6ffe966b-ae60-467e-9c3d-6600bc846a6b" containerID="1227c1421eefe6f55aa3247de691c308ac184b0d46b02d46445b43796aefa76f" exitCode=0
Oct 30 19:46:26.027101 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.027008    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2" event={"ID":"6ffe966b-ae60-467e-9c3d-6600bc846a6b","Type":"ContainerDied","Data":"1227c1421eefe6f55aa3247de691c308ac184b0d46b02d46445b43796aefa76f"}
Oct 30 19:46:26.029732 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.029233    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k" event={"ID":"eadb3435-17d0-4041-8210-3b5cfe7d50bc","Type":"ContainerStarted","Data":"7a8d4ac39d73b91755ab2adbb1605fdd8ba26c29b31aad1082eb16eadc42d0ed"}
Oct 30 19:46:26.033393 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.033326    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"]
Oct 30 19:46:26.061944 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.061822    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-xxvxj"]
Oct 30 19:46:26.074263 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.070388    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"]
Oct 30 19:46:26.079784 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.079731    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-2405fb-openshift-default-556d785b45-qthgv"]
Oct 30 19:46:26.082164 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.081813    4524 scope.go:122] "RemoveContainer" containerID="1f13892b8c237be127bbc24d152df5bd364541da9700573024572b9770672eb5"
Oct 30 19:46:26.082556 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:46:26.082405    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f13892b8c237be127bbc24d152df5bd364541da9700573024572b9770672eb5\": container with ID starting with 1f13892b8c237be127bbc24d152df5bd364541da9700573024572b9770672eb5 not found: ID does not exist" containerID="1f13892b8c237be127bbc24d152df5bd364541da9700573024572b9770672eb5"
Oct 30 19:46:26.082556 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.082441    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f13892b8c237be127bbc24d152df5bd364541da9700573024572b9770672eb5"} err="failed to get container status \"1f13892b8c237be127bbc24d152df5bd364541da9700573024572b9770672eb5\": rpc error: code = NotFound desc = could not find container \"1f13892b8c237be127bbc24d152df5bd364541da9700573024572b9770672eb5\": container with ID starting with 1f13892b8c237be127bbc24d152df5bd364541da9700573024572b9770672eb5 not found: ID does not exist"
Oct 30 19:46:26.082556 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.082465    4524 scope.go:122] "RemoveContainer" containerID="ff4d2730b5a342a1814c559e5c7442f882394ca42b8825dada709b95477ca051"
Oct 30 19:46:26.104458 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.104345    4524 scope.go:122] "RemoveContainer" containerID="ff4d2730b5a342a1814c559e5c7442f882394ca42b8825dada709b95477ca051"
Oct 30 19:46:26.104943 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:46:26.104862    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff4d2730b5a342a1814c559e5c7442f882394ca42b8825dada709b95477ca051\": container with ID starting with ff4d2730b5a342a1814c559e5c7442f882394ca42b8825dada709b95477ca051 not found: ID does not exist" containerID="ff4d2730b5a342a1814c559e5c7442f882394ca42b8825dada709b95477ca051"
Oct 30 19:46:26.104943 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.104898    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff4d2730b5a342a1814c559e5c7442f882394ca42b8825dada709b95477ca051"} err="failed to get container status \"ff4d2730b5a342a1814c559e5c7442f882394ca42b8825dada709b95477ca051\": rpc error: code = NotFound desc = could not find container \"ff4d2730b5a342a1814c559e5c7442f882394ca42b8825dada709b95477ca051\": container with ID starting with ff4d2730b5a342a1814c559e5c7442f882394ca42b8825dada709b95477ca051 not found: ID does not exist"
Oct 30 19:46:26.326236 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.323633    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"
Oct 30 19:46:26.341100 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.340965    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a26c2511-7fe6-4a60-86f8-781c17fac337-kube-api-access-h277g\" (UniqueName: \"kubernetes.io/projected/a26c2511-7fe6-4a60-86f8-781c17fac337-kube-api-access-h277g\") pod \"a26c2511-7fe6-4a60-86f8-781c17fac337\" (UID: \"a26c2511-7fe6-4a60-86f8-781c17fac337\") "
Oct 30 19:46:26.341100 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.341037    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a26c2511-7fe6-4a60-86f8-781c17fac337-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a26c2511-7fe6-4a60-86f8-781c17fac337-openshift-service-ca\") pod \"a26c2511-7fe6-4a60-86f8-781c17fac337\" (UID: \"a26c2511-7fe6-4a60-86f8-781c17fac337\") "
Oct 30 19:46:26.341361 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.341137    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a26c2511-7fe6-4a60-86f8-781c17fac337-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a26c2511-7fe6-4a60-86f8-781c17fac337-maas-api-tls\") pod \"a26c2511-7fe6-4a60-86f8-781c17fac337\" (UID: \"a26c2511-7fe6-4a60-86f8-781c17fac337\") "
Oct 30 19:46:26.341361 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.341253    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a26c2511-7fe6-4a60-86f8-781c17fac337-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a26c2511-7fe6-4a60-86f8-781c17fac337-metrics-certs\") pod \"a26c2511-7fe6-4a60-86f8-781c17fac337\" (UID: \"a26c2511-7fe6-4a60-86f8-781c17fac337\") "
Oct 30 19:46:26.376755 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.369741    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a26c2511-7fe6-4a60-86f8-781c17fac337-openshift-service-ca" pod "a26c2511-7fe6-4a60-86f8-781c17fac337" (UID: "a26c2511-7fe6-4a60-86f8-781c17fac337"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:26.388229 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.385359    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2"
Oct 30 19:46:26.417413 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.417355    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a26c2511-7fe6-4a60-86f8-781c17fac337-metrics-certs" pod "a26c2511-7fe6-4a60-86f8-781c17fac337" (UID: "a26c2511-7fe6-4a60-86f8-781c17fac337"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:26.417923 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.417892    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0ad9005d-b179-48d1-8a9f-67fff6724a6f" path="/var/lib/kubelet/pods/0ad9005d-b179-48d1-8a9f-67fff6724a6f/volumes"
Oct 30 19:46:26.418957 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.418936    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" path="/var/lib/kubelet/pods/f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f/volumes"
Oct 30 19:46:26.431193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.431130    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a26c2511-7fe6-4a60-86f8-781c17fac337-maas-api-tls" pod "a26c2511-7fe6-4a60-86f8-781c17fac337" (UID: "a26c2511-7fe6-4a60-86f8-781c17fac337"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:26.433417 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.433378    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a26c2511-7fe6-4a60-86f8-781c17fac337-kube-api-access-h277g" pod "a26c2511-7fe6-4a60-86f8-781c17fac337" (UID: "a26c2511-7fe6-4a60-86f8-781c17fac337"). InnerVolumeSpecName "kube-api-access-h277g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:26.446210 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.444788    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6ffe966b-ae60-467e-9c3d-6600bc846a6b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ffe966b-ae60-467e-9c3d-6600bc846a6b-plugins-config-volume\") pod \"6ffe966b-ae60-467e-9c3d-6600bc846a6b\" (UID: \"6ffe966b-ae60-467e-9c3d-6600bc846a6b\") "
Oct 30 19:46:26.446210 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.444928    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6ffe966b-ae60-467e-9c3d-6600bc846a6b-kube-api-access-b5fkw\" (UniqueName: \"kubernetes.io/projected/6ffe966b-ae60-467e-9c3d-6600bc846a6b-kube-api-access-b5fkw\") pod \"6ffe966b-ae60-467e-9c3d-6600bc846a6b\" (UID: \"6ffe966b-ae60-467e-9c3d-6600bc846a6b\") "
Oct 30 19:46:26.446210 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.445230    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ffe966b-ae60-467e-9c3d-6600bc846a6b-plugins-config-volume" pod "6ffe966b-ae60-467e-9c3d-6600bc846a6b" (UID: "6ffe966b-ae60-467e-9c3d-6600bc846a6b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:26.446210 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.445581    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a26c2511-7fe6-4a60-86f8-781c17fac337-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:26.446210 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.445596    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a26c2511-7fe6-4a60-86f8-781c17fac337-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:26.446210 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.445606    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ffe966b-ae60-467e-9c3d-6600bc846a6b-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:26.446210 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.445617    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h277g\" (UniqueName: \"kubernetes.io/projected/a26c2511-7fe6-4a60-86f8-781c17fac337-kube-api-access-h277g\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:26.446210 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.445625    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a26c2511-7fe6-4a60-86f8-781c17fac337-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:26.448582 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.448563    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ffe966b-ae60-467e-9c3d-6600bc846a6b-kube-api-access-b5fkw" pod "6ffe966b-ae60-467e-9c3d-6600bc846a6b" (UID: "6ffe966b-ae60-467e-9c3d-6600bc846a6b"). InnerVolumeSpecName "kube-api-access-b5fkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:26.546734 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:26.546704    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5fkw\" (UniqueName: \"kubernetes.io/projected/6ffe966b-ae60-467e-9c3d-6600bc846a6b-kube-api-access-b5fkw\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:27.037863 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:27.037830    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"
Oct 30 19:46:27.038391 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:27.037860    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc" event={"ID":"a26c2511-7fe6-4a60-86f8-781c17fac337","Type":"ContainerDied","Data":"388b3a5d4fd343a47af1be40000c054b7b6d9ad09f5af892e344274fda4e35ad"}
Oct 30 19:46:27.038391 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:27.037924    4524 scope.go:122] "RemoveContainer" containerID="ca99ebc94faea5f62cdb83ed0be289349a2db2368e4268c297f0d712c6ae3452"
Oct 30 19:46:27.039545 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:27.039523    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2"
Oct 30 19:46:27.039545 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:27.039535    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2" event={"ID":"6ffe966b-ae60-467e-9c3d-6600bc846a6b","Type":"ContainerDied","Data":"58f7cbd442de56d8957fdfaf62d7e6404b71f3c4950f1a73dde2858281d8ea93"}
Oct 30 19:46:27.041622 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:27.041553    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k" event={"ID":"eadb3435-17d0-4041-8210-3b5cfe7d50bc","Type":"ContainerStarted","Data":"5cc0bfbb90853618f2fc33c117a7ad1ffc8a00a03d01069aa58da6683d4e26cc"}
Oct 30 19:46:27.041730 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:27.041619    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"
Oct 30 19:46:27.051458 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:27.051438    4524 scope.go:122] "RemoveContainer" containerID="1227c1421eefe6f55aa3247de691c308ac184b0d46b02d46445b43796aefa76f"
Oct 30 19:46:27.072406 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:27.072359    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k" podStartSLOduration=2.478327004 podStartE2EDuration="3.072343118s" totalImagesPullingTime="594.016114ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:46:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:46:27.07192209 +0000 UTC m=+2418.812913206" watchObservedRunningTime="2026-10-30 19:46:27.072343118 +0000 UTC m=+2418.813334199"
Oct 30 19:46:27.091252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:27.091224    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2"]
Oct 30 19:46:27.095852 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:27.095825    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-e50648-6d6956b455-hrvd2"]
Oct 30 19:46:27.109328 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:27.109268    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"]
Oct 30 19:46:27.113061 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:27.113042    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-e50648-6cc77f45-fqwnc"]
Oct 30 19:46:28.421194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:28.411046    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6ffe966b-ae60-467e-9c3d-6600bc846a6b" path="/var/lib/kubelet/pods/6ffe966b-ae60-467e-9c3d-6600bc846a6b/volumes"
Oct 30 19:46:28.421194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:28.412013    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a26c2511-7fe6-4a60-86f8-781c17fac337" path="/var/lib/kubelet/pods/a26c2511-7fe6-4a60-86f8-781c17fac337/volumes"
Oct 30 19:46:30.396864 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:30.396828    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s"]
Oct 30 19:46:32.507073 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:32.506951    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5" podUID="5355e2dd-ba58-4602-bedc-44a2979762e1" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:33.059271 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:33.059236    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"
Oct 30 19:46:33.514192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:33.513997    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d" podUID="34a1d722-a8a4-45e0-a5e4-6306c70eb7bb" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:35.090641 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:35.090597    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"]
Oct 30 19:46:35.091079 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:35.090836    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg" podUID="98c13946-1bdd-4129-9b78-88c56659f432" containerName="istio-proxy" containerID="cri-o://3b12f1a25d91ac5434e5f60ff5620e1cd65ad22947861557ababa80e1817568c" gracePeriod=30
Oct 30 19:46:35.398262 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:35.398108    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s" podUID="050d3ce5-caa8-4f50-8328-bef71892924b" containerName="payload-processing" containerID="cri-o://73fae5e4e2af401846687690f9896c2b2158ba01c48bf89fdc5013b73e6f017d" gracePeriod=25
Oct 30 19:46:35.815842 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:35.815755    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s" podUID="050d3ce5-caa8-4f50-8328-bef71892924b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:37.187791 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:37.187741    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"]
Oct 30 19:46:37.188284 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:37.188039    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v" podUID="c23cec30-c930-46cc-a1c0-2c4f347dfcd7" containerName="maas-api" containerID="cri-o://652679f7f0f151f7210f9842af5908b7a81204141bf0d136dbc66975c7935f8d" gracePeriod=30
Oct 30 19:46:37.710243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:37.710215    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"
Oct 30 19:46:37.730424 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:37.730396    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-kube-api-access-qbxhm\" (UniqueName: \"kubernetes.io/projected/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-kube-api-access-qbxhm\") pod \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\" (UID: \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\") "
Oct 30 19:46:37.730598 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:37.730435    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-openshift-service-ca\") pod \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\" (UID: \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\") "
Oct 30 19:46:37.730598 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:37.730511    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-metrics-certs\") pod \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\" (UID: \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\") "
Oct 30 19:46:37.730725 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:37.730623    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-maas-api-tls\") pod \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\" (UID: \"c23cec30-c930-46cc-a1c0-2c4f347dfcd7\") "
Oct 30 19:46:37.731061 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:37.731036    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-openshift-service-ca" pod "c23cec30-c930-46cc-a1c0-2c4f347dfcd7" (UID: "c23cec30-c930-46cc-a1c0-2c4f347dfcd7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:37.731560 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:37.731540    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:37.733927 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:37.733890    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-metrics-certs" pod "c23cec30-c930-46cc-a1c0-2c4f347dfcd7" (UID: "c23cec30-c930-46cc-a1c0-2c4f347dfcd7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:37.734031 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:37.734009    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-kube-api-access-qbxhm" pod "c23cec30-c930-46cc-a1c0-2c4f347dfcd7" (UID: "c23cec30-c930-46cc-a1c0-2c4f347dfcd7"). InnerVolumeSpecName "kube-api-access-qbxhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:37.736045 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:37.736008    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-maas-api-tls" pod "c23cec30-c930-46cc-a1c0-2c4f347dfcd7" (UID: "c23cec30-c930-46cc-a1c0-2c4f347dfcd7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:37.832777 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:37.832740    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:37.832777 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:37.832775    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:37.832777 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:37.832785    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbxhm\" (UniqueName: \"kubernetes.io/projected/c23cec30-c930-46cc-a1c0-2c4f347dfcd7-kube-api-access-qbxhm\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:38.312015 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:38.311976    4524 generic.go:356] "Generic (PLEG): container finished" podID="c23cec30-c930-46cc-a1c0-2c4f347dfcd7" containerID="652679f7f0f151f7210f9842af5908b7a81204141bf0d136dbc66975c7935f8d" exitCode=0
Oct 30 19:46:38.312525 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:38.312034    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"
Oct 30 19:46:38.312525 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:38.312059    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v" event={"ID":"c23cec30-c930-46cc-a1c0-2c4f347dfcd7","Type":"ContainerDied","Data":"652679f7f0f151f7210f9842af5908b7a81204141bf0d136dbc66975c7935f8d"}
Oct 30 19:46:38.312525 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:38.312111    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v" event={"ID":"c23cec30-c930-46cc-a1c0-2c4f347dfcd7","Type":"ContainerDied","Data":"1889a1e5e35aedd4518e2e3d3e689a6e1200641abe03a2c1ccf17b4dbf753e4e"}
Oct 30 19:46:38.312525 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:38.312140    4524 scope.go:122] "RemoveContainer" containerID="652679f7f0f151f7210f9842af5908b7a81204141bf0d136dbc66975c7935f8d"
Oct 30 19:46:38.324607 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:38.324587    4524 scope.go:122] "RemoveContainer" containerID="652679f7f0f151f7210f9842af5908b7a81204141bf0d136dbc66975c7935f8d"
Oct 30 19:46:38.324899 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:46:38.324880    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"652679f7f0f151f7210f9842af5908b7a81204141bf0d136dbc66975c7935f8d\": container with ID starting with 652679f7f0f151f7210f9842af5908b7a81204141bf0d136dbc66975c7935f8d not found: ID does not exist" containerID="652679f7f0f151f7210f9842af5908b7a81204141bf0d136dbc66975c7935f8d"
Oct 30 19:46:38.324969 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:38.324910    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"652679f7f0f151f7210f9842af5908b7a81204141bf0d136dbc66975c7935f8d"} err="failed to get container status \"652679f7f0f151f7210f9842af5908b7a81204141bf0d136dbc66975c7935f8d\": rpc error: code = NotFound desc = could not find container \"652679f7f0f151f7210f9842af5908b7a81204141bf0d136dbc66975c7935f8d\": container with ID starting with 652679f7f0f151f7210f9842af5908b7a81204141bf0d136dbc66975c7935f8d not found: ID does not exist"
Oct 30 19:46:38.338628 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:38.338605    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"]
Oct 30 19:46:38.345719 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:38.345698    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-1e7123-bbb4955bf-m9v4v"]
Oct 30 19:46:38.397554 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:38.397517    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c23cec30-c930-46cc-a1c0-2c4f347dfcd7" path="/var/lib/kubelet/pods/c23cec30-c930-46cc-a1c0-2c4f347dfcd7/volumes"
Oct 30 19:46:40.359571 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.356936    4524 generic.go:356] "Generic (PLEG): container finished" podID="5355e2dd-ba58-4602-bedc-44a2979762e1" containerID="77e7d5836186b50b3e4ebbe4a881cb0b05e5d69511a784fd468220fee8ef5e59" exitCode=0
Oct 30 19:46:40.359571 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.357054    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5" event={"ID":"5355e2dd-ba58-4602-bedc-44a2979762e1","Type":"ContainerDied","Data":"77e7d5836186b50b3e4ebbe4a881cb0b05e5d69511a784fd468220fee8ef5e59"}
Oct 30 19:46:40.359571 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.359000    4524 generic.go:356] "Generic (PLEG): container finished" podID="98c13946-1bdd-4129-9b78-88c56659f432" containerID="3b12f1a25d91ac5434e5f60ff5620e1cd65ad22947861557ababa80e1817568c" exitCode=0
Oct 30 19:46:40.359571 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.359183    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg" event={"ID":"98c13946-1bdd-4129-9b78-88c56659f432","Type":"ContainerDied","Data":"3b12f1a25d91ac5434e5f60ff5620e1cd65ad22947861557ababa80e1817568c"}
Oct 30 19:46:40.359571 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.359222    4524 scope.go:122] "RemoveContainer" containerID="69cbdc3aeed23a0bd335cc14eb2a3e7e5b7894532dd5b047b2ab9eda6f9aae49"
Oct 30 19:46:40.363814 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.363771    4524 generic.go:356] "Generic (PLEG): container finished" podID="34a1d722-a8a4-45e0-a5e4-6306c70eb7bb" containerID="617aca740b732e2809707bb638e6595aba570469fbc6e61a7ea61ca149679b2f" exitCode=0
Oct 30 19:46:40.365324 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.365277    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d" event={"ID":"34a1d722-a8a4-45e0-a5e4-6306c70eb7bb","Type":"ContainerDied","Data":"617aca740b732e2809707bb638e6595aba570469fbc6e61a7ea61ca149679b2f"}
Oct 30 19:46:40.453003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.452966    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:46:40.530542 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.526609    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-workload-socket\") pod \"98c13946-1bdd-4129-9b78-88c56659f432\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") "
Oct 30 19:46:40.530542 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.526713    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/98c13946-1bdd-4129-9b78-88c56659f432-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98c13946-1bdd-4129-9b78-88c56659f432-istiod-ca-cert\") pod \"98c13946-1bdd-4129-9b78-88c56659f432\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") "
Oct 30 19:46:40.530542 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.526791    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-credential-socket\") pod \"98c13946-1bdd-4129-9b78-88c56659f432\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") "
Oct 30 19:46:40.530542 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.526833    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98c13946-1bdd-4129-9b78-88c56659f432-istio-token\" (UniqueName: \"kubernetes.io/projected/98c13946-1bdd-4129-9b78-88c56659f432-istio-token\") pod \"98c13946-1bdd-4129-9b78-88c56659f432\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") "
Oct 30 19:46:40.530542 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.526974    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98c13946-1bdd-4129-9b78-88c56659f432-kube-api-access-8bxgr\" (UniqueName: \"kubernetes.io/projected/98c13946-1bdd-4129-9b78-88c56659f432-kube-api-access-8bxgr\") pod \"98c13946-1bdd-4129-9b78-88c56659f432\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") "
Oct 30 19:46:40.530542 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.527025    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-workload-certs\") pod \"98c13946-1bdd-4129-9b78-88c56659f432\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") "
Oct 30 19:46:40.530542 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.527053    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-istio-envoy\") pod \"98c13946-1bdd-4129-9b78-88c56659f432\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") "
Oct 30 19:46:40.530542 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.527187    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-istio-data\") pod \"98c13946-1bdd-4129-9b78-88c56659f432\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") "
Oct 30 19:46:40.530542 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.527226    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/98c13946-1bdd-4129-9b78-88c56659f432-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98c13946-1bdd-4129-9b78-88c56659f432-istio-podinfo\") pod \"98c13946-1bdd-4129-9b78-88c56659f432\" (UID: \"98c13946-1bdd-4129-9b78-88c56659f432\") "
Oct 30 19:46:40.530542 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.529605    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98c13946-1bdd-4129-9b78-88c56659f432-istiod-ca-cert" pod "98c13946-1bdd-4129-9b78-88c56659f432" (UID: "98c13946-1bdd-4129-9b78-88c56659f432"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:40.530542 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.529837    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-workload-socket" pod "98c13946-1bdd-4129-9b78-88c56659f432" (UID: "98c13946-1bdd-4129-9b78-88c56659f432"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:40.530542 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.529993    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-workload-certs" pod "98c13946-1bdd-4129-9b78-88c56659f432" (UID: "98c13946-1bdd-4129-9b78-88c56659f432"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:40.534713 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.532424    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98c13946-1bdd-4129-9b78-88c56659f432-istio-token" pod "98c13946-1bdd-4129-9b78-88c56659f432" (UID: "98c13946-1bdd-4129-9b78-88c56659f432"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:40.534713 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.532857    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-credential-socket" pod "98c13946-1bdd-4129-9b78-88c56659f432" (UID: "98c13946-1bdd-4129-9b78-88c56659f432"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:40.534713 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.533415    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-istio-data" pod "98c13946-1bdd-4129-9b78-88c56659f432" (UID: "98c13946-1bdd-4129-9b78-88c56659f432"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:40.538336 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.535646    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98c13946-1bdd-4129-9b78-88c56659f432-kube-api-access-8bxgr" pod "98c13946-1bdd-4129-9b78-88c56659f432" (UID: "98c13946-1bdd-4129-9b78-88c56659f432"). InnerVolumeSpecName "kube-api-access-8bxgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:40.538336 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.536027    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-istio-envoy" pod "98c13946-1bdd-4129-9b78-88c56659f432" (UID: "98c13946-1bdd-4129-9b78-88c56659f432"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:40.538336 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.536325    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/98c13946-1bdd-4129-9b78-88c56659f432-istio-podinfo" pod "98c13946-1bdd-4129-9b78-88c56659f432" (UID: "98c13946-1bdd-4129-9b78-88c56659f432"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:46:40.555518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.555427    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5"
Oct 30 19:46:40.560913 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.560889    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d"
Oct 30 19:46:40.629126 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.629052    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5355e2dd-ba58-4602-bedc-44a2979762e1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5355e2dd-ba58-4602-bedc-44a2979762e1-plugins-config-volume\") pod \"5355e2dd-ba58-4602-bedc-44a2979762e1\" (UID: \"5355e2dd-ba58-4602-bedc-44a2979762e1\") "
Oct 30 19:46:40.629126 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.629095    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34a1d722-a8a4-45e0-a5e4-6306c70eb7bb-kube-api-access-n65x9\" (UniqueName: \"kubernetes.io/projected/34a1d722-a8a4-45e0-a5e4-6306c70eb7bb-kube-api-access-n65x9\") pod \"34a1d722-a8a4-45e0-a5e4-6306c70eb7bb\" (UID: \"34a1d722-a8a4-45e0-a5e4-6306c70eb7bb\") "
Oct 30 19:46:40.629126 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.629123    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/34a1d722-a8a4-45e0-a5e4-6306c70eb7bb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34a1d722-a8a4-45e0-a5e4-6306c70eb7bb-plugins-config-volume\") pod \"34a1d722-a8a4-45e0-a5e4-6306c70eb7bb\" (UID: \"34a1d722-a8a4-45e0-a5e4-6306c70eb7bb\") "
Oct 30 19:46:40.629423 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.629286    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5355e2dd-ba58-4602-bedc-44a2979762e1-kube-api-access-7j958\" (UniqueName: \"kubernetes.io/projected/5355e2dd-ba58-4602-bedc-44a2979762e1-kube-api-access-7j958\") pod \"5355e2dd-ba58-4602-bedc-44a2979762e1\" (UID: \"5355e2dd-ba58-4602-bedc-44a2979762e1\") "
Oct 30 19:46:40.629558 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.629535    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34a1d722-a8a4-45e0-a5e4-6306c70eb7bb-plugins-config-volume" pod "34a1d722-a8a4-45e0-a5e4-6306c70eb7bb" (UID: "34a1d722-a8a4-45e0-a5e4-6306c70eb7bb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:40.629673 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.629537    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5355e2dd-ba58-4602-bedc-44a2979762e1-plugins-config-volume" pod "5355e2dd-ba58-4602-bedc-44a2979762e1" (UID: "5355e2dd-ba58-4602-bedc-44a2979762e1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:40.630074 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.630050    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:40.630241 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.630225    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:40.630314 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.630246    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:40.630314 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.630261    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/98c13946-1bdd-4129-9b78-88c56659f432-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:40.630314 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.630275    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:40.630314 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.630290    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/98c13946-1bdd-4129-9b78-88c56659f432-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:40.630314 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.630303    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/98c13946-1bdd-4129-9b78-88c56659f432-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:40.630506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.630317    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/98c13946-1bdd-4129-9b78-88c56659f432-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:40.630506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.630333    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5355e2dd-ba58-4602-bedc-44a2979762e1-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:40.630506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.630349    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34a1d722-a8a4-45e0-a5e4-6306c70eb7bb-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:40.630506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.630363    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bxgr\" (UniqueName: \"kubernetes.io/projected/98c13946-1bdd-4129-9b78-88c56659f432-kube-api-access-8bxgr\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:40.663787 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.662736    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34a1d722-a8a4-45e0-a5e4-6306c70eb7bb-kube-api-access-n65x9" pod "34a1d722-a8a4-45e0-a5e4-6306c70eb7bb" (UID: "34a1d722-a8a4-45e0-a5e4-6306c70eb7bb"). InnerVolumeSpecName "kube-api-access-n65x9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:40.694456 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.694415    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5355e2dd-ba58-4602-bedc-44a2979762e1-kube-api-access-7j958" pod "5355e2dd-ba58-4602-bedc-44a2979762e1" (UID: "5355e2dd-ba58-4602-bedc-44a2979762e1"). InnerVolumeSpecName "kube-api-access-7j958". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:40.733038 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.732993    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n65x9\" (UniqueName: \"kubernetes.io/projected/34a1d722-a8a4-45e0-a5e4-6306c70eb7bb-kube-api-access-n65x9\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:40.733038 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:40.733038    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7j958\" (UniqueName: \"kubernetes.io/projected/5355e2dd-ba58-4602-bedc-44a2979762e1-kube-api-access-7j958\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:41.375187 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:41.375096    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d"
Oct 30 19:46:41.375671 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:41.375149    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d" event={"ID":"34a1d722-a8a4-45e0-a5e4-6306c70eb7bb","Type":"ContainerDied","Data":"265f1b8d6db3a93255c112692d0cbf87b0474c151d991158a15294ba9ea9cb51"}
Oct 30 19:46:41.375671 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:41.375269    4524 scope.go:122] "RemoveContainer" containerID="617aca740b732e2809707bb638e6595aba570469fbc6e61a7ea61ca149679b2f"
Oct 30 19:46:41.377320 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:41.377297    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5" event={"ID":"5355e2dd-ba58-4602-bedc-44a2979762e1","Type":"ContainerDied","Data":"9f1bedacef16f0852f3be9b14b41dac5933a987ca943e33bb7c7ce3ecc95b0e8"}
Oct 30 19:46:41.377428 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:41.377384    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5"
Oct 30 19:46:41.382222 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:41.381603    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"
Oct 30 19:46:41.382222 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:41.381609    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg" event={"ID":"98c13946-1bdd-4129-9b78-88c56659f432","Type":"ContainerDied","Data":"84e61d8d3efc7028dbfa22b50a7ffd08676fc800734e00ec9d321a40afdd62fb"}
Oct 30 19:46:41.392025 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:41.392006    4524 scope.go:122] "RemoveContainer" containerID="77e7d5836186b50b3e4ebbe4a881cb0b05e5d69511a784fd468220fee8ef5e59"
Oct 30 19:46:41.403477 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:41.403431    4524 scope.go:122] "RemoveContainer" containerID="3b12f1a25d91ac5434e5f60ff5620e1cd65ad22947861557ababa80e1817568c"
Oct 30 19:46:41.414798 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:41.414777    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"]
Oct 30 19:46:41.424284 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:41.424263    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-e50648-openshift-default-5b7dcc895f-m9fwg"]
Oct 30 19:46:41.436683 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:41.436662    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5"]
Oct 30 19:46:41.442150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:41.442125    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-cfa4bb-9f6f8fcd4-nqcn5"]
Oct 30 19:46:41.451103 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:41.451083    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d"]
Oct 30 19:46:41.456282 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:41.456265    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-cfa4bb-5795d58567-jhk7d"]
Oct 30 19:46:42.423180 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:42.423148    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="34a1d722-a8a4-45e0-a5e4-6306c70eb7bb" path="/var/lib/kubelet/pods/34a1d722-a8a4-45e0-a5e4-6306c70eb7bb/volumes"
Oct 30 19:46:42.423884 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:42.423862    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5355e2dd-ba58-4602-bedc-44a2979762e1" path="/var/lib/kubelet/pods/5355e2dd-ba58-4602-bedc-44a2979762e1/volumes"
Oct 30 19:46:42.424435 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:42.424416    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="98c13946-1bdd-4129-9b78-88c56659f432" path="/var/lib/kubelet/pods/98c13946-1bdd-4129-9b78-88c56659f432/volumes"
Oct 30 19:46:43.173237 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.173203    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"]
Oct 30 19:46:43.174347 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174318    4524 cpu_manager.go:414] "containerMap: removing container" podUID="c23cec30-c930-46cc-a1c0-2c4f347dfcd7" containerName="maas-api"
Oct 30 19:46:43.174347 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174346    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c23cec30-c930-46cc-a1c0-2c4f347dfcd7" containerName="maas-api"
Oct 30 19:46:43.174552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174369    4524 cpu_manager.go:414] "containerMap: removing container" podUID="98c13946-1bdd-4129-9b78-88c56659f432" containerName="istio-proxy"
Oct 30 19:46:43.174552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174376    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="98c13946-1bdd-4129-9b78-88c56659f432" containerName="istio-proxy"
Oct 30 19:46:43.174552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174400    4524 cpu_manager.go:414] "containerMap: removing container" podUID="5355e2dd-ba58-4602-bedc-44a2979762e1" containerName="payload-processing"
Oct 30 19:46:43.174552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174409    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5355e2dd-ba58-4602-bedc-44a2979762e1" containerName="payload-processing"
Oct 30 19:46:43.174552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174424    4524 cpu_manager.go:414] "containerMap: removing container" podUID="a26c2511-7fe6-4a60-86f8-781c17fac337" containerName="maas-api"
Oct 30 19:46:43.174552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174429    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a26c2511-7fe6-4a60-86f8-781c17fac337" containerName="maas-api"
Oct 30 19:46:43.174552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174438    4524 cpu_manager.go:414] "containerMap: removing container" podUID="f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" containerName="istio-proxy"
Oct 30 19:46:43.174552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174442    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" containerName="istio-proxy"
Oct 30 19:46:43.174552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174458    4524 cpu_manager.go:414] "containerMap: removing container" podUID="0ad9005d-b179-48d1-8a9f-67fff6724a6f" containerName="istio-proxy"
Oct 30 19:46:43.174552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174465    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0ad9005d-b179-48d1-8a9f-67fff6724a6f" containerName="istio-proxy"
Oct 30 19:46:43.174552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174486    4524 cpu_manager.go:414] "containerMap: removing container" podUID="6ffe966b-ae60-467e-9c3d-6600bc846a6b" containerName="payload-pre-processing"
Oct 30 19:46:43.174552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174493    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6ffe966b-ae60-467e-9c3d-6600bc846a6b" containerName="payload-pre-processing"
Oct 30 19:46:43.174552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174520    4524 cpu_manager.go:414] "containerMap: removing container" podUID="34a1d722-a8a4-45e0-a5e4-6306c70eb7bb" containerName="payload-pre-processing"
Oct 30 19:46:43.174552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174528    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34a1d722-a8a4-45e0-a5e4-6306c70eb7bb" containerName="payload-pre-processing"
Oct 30 19:46:43.175150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174670    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="6ffe966b-ae60-467e-9c3d-6600bc846a6b" containerName="payload-pre-processing"
Oct 30 19:46:43.175150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174687    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="98c13946-1bdd-4129-9b78-88c56659f432" containerName="istio-proxy"
Oct 30 19:46:43.175150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174697    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="34a1d722-a8a4-45e0-a5e4-6306c70eb7bb" containerName="payload-pre-processing"
Oct 30 19:46:43.175150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174707    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="c23cec30-c930-46cc-a1c0-2c4f347dfcd7" containerName="maas-api"
Oct 30 19:46:43.175150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174718    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="98c13946-1bdd-4129-9b78-88c56659f432" containerName="istio-proxy"
Oct 30 19:46:43.175150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174730    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="5355e2dd-ba58-4602-bedc-44a2979762e1" containerName="payload-processing"
Oct 30 19:46:43.175150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174741    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="a26c2511-7fe6-4a60-86f8-781c17fac337" containerName="maas-api"
Oct 30 19:46:43.175150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174749    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="f95c8550-35b7-4f3d-9b2d-b0a3d8022a7f" containerName="istio-proxy"
Oct 30 19:46:43.175150 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.174759    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="0ad9005d-b179-48d1-8a9f-67fff6724a6f" containerName="istio-proxy"
Oct 30 19:46:43.201208 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.180783    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.212043 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.212010    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"]
Oct 30 19:46:43.272742 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.272712    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-workload-socket\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.272899 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.272753    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-data\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.272899 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.272781    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2775b\" (UniqueName: \"kubernetes.io/projected/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-kube-api-access-2775b\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.272985 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.272911    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-token\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.272985 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.272954    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-envoy\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.273057 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.273018    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-credential-socket\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.273094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.273079    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istiod-ca-cert\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.273129 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.273107    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-podinfo\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.273213 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.273201    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-workload-certs\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.375124 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.375092    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-workload-socket\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.375318 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.375137    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-data\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.375318 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.375213    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2775b\" (UniqueName: \"kubernetes.io/projected/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-kube-api-access-2775b\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.375318 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.375263    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-token\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.375318 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.375308    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-envoy\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.375561 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.375347    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-credential-socket\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.375561 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.375387    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istiod-ca-cert\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.375561 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.375428    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-podinfo\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.375561 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.375539    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-workload-certs\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.375776 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.375696    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-credential-socket\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.375869 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.375847    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-workload-socket\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.375952 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.375930    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-data\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.375996 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.375981    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-workload-certs\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.376123 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.376101    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istiod-ca-cert\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.378264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.378229    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-envoy\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.378583 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.378562    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-podinfo\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.386984 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.386960    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-token\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.387104 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.387088    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2775b\" (UniqueName: \"kubernetes.io/projected/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-kube-api-access-2775b\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.535959 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.535879    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:43.900955 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.887852    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"]
Oct 30 19:46:43.934434 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.934396    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:46:43.934613 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.934473    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:46:43.934613 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:43.934512    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:46:44.439497 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:44.439458    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2" event={"ID":"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457","Type":"ContainerStarted","Data":"a0c7c04ab829c65b0dc75c9b6908e40ab662c1d8d88481a8edf1e6e684138c28"}
Oct 30 19:46:44.439798 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:44.439523    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2" event={"ID":"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457","Type":"ContainerStarted","Data":"28093e303c74631ea03bd8af83c101bbb5f20314ea77a87f93a1caad51a386f0"}
Oct 30 19:46:44.466555 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:44.466506    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2" podStartSLOduration=1.466494213 podStartE2EDuration="1.466494213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:46:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:46:44.462085956 +0000 UTC m=+2436.203077036" watchObservedRunningTime="2026-10-30 19:46:44.466494213 +0000 UTC m=+2436.207485293"
Oct 30 19:46:44.536793 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:44.536759    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:45.537944 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:45.537904    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.186:15021/healthz/ready\": dial tcp 10.133.0.186:15021: connect: connection refused" start-of-body=
Oct 30 19:46:45.538358 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:45.537956    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2" podUID="0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.186:15021/healthz/ready\": dial tcp 10.133.0.186:15021: connect: connection refused"
Oct 30 19:46:45.641192 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:46:45.640897    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:46:45.816320 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:45.815949    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s" podUID="050d3ce5-caa8-4f50-8328-bef71892924b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:46.555193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:46.536508    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.186:15021/healthz/ready\": dial tcp 10.133.0.186:15021: connect: connection refused" start-of-body=
Oct 30 19:46:46.555193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:46.536568    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2" podUID="0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.186:15021/healthz/ready\": dial tcp 10.133.0.186:15021: connect: connection refused"
Oct 30 19:46:47.538197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:47.536541    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.186:15021/healthz/ready\": dial tcp 10.133.0.186:15021: connect: connection refused" start-of-body=
Oct 30 19:46:47.538197 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:47.536606    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2" podUID="0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.186:15021/healthz/ready\": dial tcp 10.133.0.186:15021: connect: connection refused"
Oct 30 19:46:48.538802 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:48.538760    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.186:15021/healthz/ready\": dial tcp 10.133.0.186:15021: connect: connection refused" start-of-body=
Oct 30 19:46:48.539291 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:48.538825    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2" podUID="0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.186:15021/healthz/ready\": dial tcp 10.133.0.186:15021: connect: connection refused"
Oct 30 19:46:49.538545 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:49.538507    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.186:15021/healthz/ready\": dial tcp 10.133.0.186:15021: connect: connection refused" start-of-body=
Oct 30 19:46:49.538946 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:49.538916    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2" podUID="0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.186:15021/healthz/ready\": dial tcp 10.133.0.186:15021: connect: connection refused"
Oct 30 19:46:50.614239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:50.541439    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.186:15021/healthz/ready\": dial tcp 10.133.0.186:15021: connect: connection refused" start-of-body=
Oct 30 19:46:50.614239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:50.541499    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2" podUID="0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.186:15021/healthz/ready\": dial tcp 10.133.0.186:15021: connect: connection refused"
Oct 30 19:46:50.948532 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:50.948449    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"]
Oct 30 19:46:50.949584 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:50.949554    4524 cpu_manager.go:414] "containerMap: removing container" podUID="98c13946-1bdd-4129-9b78-88c56659f432" containerName="istio-proxy"
Oct 30 19:46:50.949584 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:50.949583    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="98c13946-1bdd-4129-9b78-88c56659f432" containerName="istio-proxy"
Oct 30 19:46:50.956724 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:50.956702    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"
Oct 30 19:46:50.989228 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:50.968607    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"]
Oct 30 19:46:51.018496 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:51.018444    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-be6b74-serving-cert\""
Oct 30 19:46:51.118684 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:51.118653    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b541712-a45c-41a1-bfcc-fc7349f4355b-maas-api-tls\") pod \"maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2\" (UID: \"8b541712-a45c-41a1-bfcc-fc7349f4355b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"
Oct 30 19:46:51.118886 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:51.118707    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b541712-a45c-41a1-bfcc-fc7349f4355b-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2\" (UID: \"8b541712-a45c-41a1-bfcc-fc7349f4355b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"
Oct 30 19:46:51.118961 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:51.118893    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84r8n\" (UniqueName: \"kubernetes.io/projected/8b541712-a45c-41a1-bfcc-fc7349f4355b-kube-api-access-84r8n\") pod \"maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2\" (UID: \"8b541712-a45c-41a1-bfcc-fc7349f4355b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"
Oct 30 19:46:51.119021 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:51.118966    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b541712-a45c-41a1-bfcc-fc7349f4355b-metrics-certs\") pod \"maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2\" (UID: \"8b541712-a45c-41a1-bfcc-fc7349f4355b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"
Oct 30 19:46:51.220243 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:51.220112    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b541712-a45c-41a1-bfcc-fc7349f4355b-metrics-certs\") pod \"maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2\" (UID: \"8b541712-a45c-41a1-bfcc-fc7349f4355b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"
Oct 30 19:46:51.220641 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:51.220602    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b541712-a45c-41a1-bfcc-fc7349f4355b-maas-api-tls\") pod \"maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2\" (UID: \"8b541712-a45c-41a1-bfcc-fc7349f4355b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"
Oct 30 19:46:51.220833 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:51.220803    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b541712-a45c-41a1-bfcc-fc7349f4355b-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2\" (UID: \"8b541712-a45c-41a1-bfcc-fc7349f4355b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"
Oct 30 19:46:51.221972 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:51.221944    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b541712-a45c-41a1-bfcc-fc7349f4355b-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2\" (UID: \"8b541712-a45c-41a1-bfcc-fc7349f4355b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"
Oct 30 19:46:51.223482 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:51.223442    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84r8n\" (UniqueName: \"kubernetes.io/projected/8b541712-a45c-41a1-bfcc-fc7349f4355b-kube-api-access-84r8n\") pod \"maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2\" (UID: \"8b541712-a45c-41a1-bfcc-fc7349f4355b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"
Oct 30 19:46:51.235385 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:51.235359    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b541712-a45c-41a1-bfcc-fc7349f4355b-maas-api-tls\") pod \"maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2\" (UID: \"8b541712-a45c-41a1-bfcc-fc7349f4355b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"
Oct 30 19:46:51.239702 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:51.236825    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84r8n\" (UniqueName: \"kubernetes.io/projected/8b541712-a45c-41a1-bfcc-fc7349f4355b-kube-api-access-84r8n\") pod \"maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2\" (UID: \"8b541712-a45c-41a1-bfcc-fc7349f4355b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"
Oct 30 19:46:51.240740 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:51.240717    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b541712-a45c-41a1-bfcc-fc7349f4355b-metrics-certs\") pod \"maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2\" (UID: \"8b541712-a45c-41a1-bfcc-fc7349f4355b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"
Oct 30 19:46:51.327448 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:51.327408    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"
Oct 30 19:46:51.552231 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:51.550436    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:51.552231 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:51.551440    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:51.553896 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:51.553854    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:46:51.675193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:51.667873    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"]
Oct 30 19:46:52.664276 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:52.664233    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2" event={"ID":"8b541712-a45c-41a1-bfcc-fc7349f4355b","Type":"ContainerStarted","Data":"8f26347c52826768460bb3361f4ae45f59cbf74d26018b4db32e485883331751"}
Oct 30 19:46:52.664474 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:52.664311    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2" event={"ID":"8b541712-a45c-41a1-bfcc-fc7349f4355b","Type":"ContainerStarted","Data":"96af1adc5d91610f226782d19db3c7882ccba561ec2f133d91b9cbef28c20e56"}
Oct 30 19:46:52.664474 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:52.664355    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"
Oct 30 19:46:52.688618 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:52.688567    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2" podStartSLOduration=2.149719714 podStartE2EDuration="2.688554152s" totalImagesPullingTime="538.834438ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:46:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:46:52.683218923 +0000 UTC m=+2444.424210004" watchObservedRunningTime="2026-10-30 19:46:52.688554152 +0000 UTC m=+2444.429545230"
Oct 30 19:46:53.718107 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:53.717604    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"]
Oct 30 19:46:53.718619 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:53.717887    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr" podUID="fbb268b8-644a-41c8-a78b-2779914d0e9d" containerName="maas-api" containerID="cri-o://fdcf6fb1b03028e373db4d39d4a5f86ab67afd01659d86a44a9b934e65a6c131" gracePeriod=30
Oct 30 19:46:54.273123 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.273096    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"
Oct 30 19:46:54.385976 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.385944    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbb268b8-644a-41c8-a78b-2779914d0e9d-kube-api-access-4dg4v\" (UniqueName: \"kubernetes.io/projected/fbb268b8-644a-41c8-a78b-2779914d0e9d-kube-api-access-4dg4v\") pod \"fbb268b8-644a-41c8-a78b-2779914d0e9d\" (UID: \"fbb268b8-644a-41c8-a78b-2779914d0e9d\") "
Oct 30 19:46:54.386156 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.386126    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fbb268b8-644a-41c8-a78b-2779914d0e9d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbb268b8-644a-41c8-a78b-2779914d0e9d-openshift-service-ca\") pod \"fbb268b8-644a-41c8-a78b-2779914d0e9d\" (UID: \"fbb268b8-644a-41c8-a78b-2779914d0e9d\") "
Oct 30 19:46:54.386271 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.386256    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fbb268b8-644a-41c8-a78b-2779914d0e9d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb268b8-644a-41c8-a78b-2779914d0e9d-metrics-certs\") pod \"fbb268b8-644a-41c8-a78b-2779914d0e9d\" (UID: \"fbb268b8-644a-41c8-a78b-2779914d0e9d\") "
Oct 30 19:46:54.386435 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.386412    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fbb268b8-644a-41c8-a78b-2779914d0e9d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbb268b8-644a-41c8-a78b-2779914d0e9d-maas-api-tls\") pod \"fbb268b8-644a-41c8-a78b-2779914d0e9d\" (UID: \"fbb268b8-644a-41c8-a78b-2779914d0e9d\") "
Oct 30 19:46:54.386532 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.386484    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbb268b8-644a-41c8-a78b-2779914d0e9d-openshift-service-ca" pod "fbb268b8-644a-41c8-a78b-2779914d0e9d" (UID: "fbb268b8-644a-41c8-a78b-2779914d0e9d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:54.386948 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.386925    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbb268b8-644a-41c8-a78b-2779914d0e9d-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:54.388975 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.388952    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbb268b8-644a-41c8-a78b-2779914d0e9d-metrics-certs" pod "fbb268b8-644a-41c8-a78b-2779914d0e9d" (UID: "fbb268b8-644a-41c8-a78b-2779914d0e9d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:54.389086 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.389014    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbb268b8-644a-41c8-a78b-2779914d0e9d-kube-api-access-4dg4v" pod "fbb268b8-644a-41c8-a78b-2779914d0e9d" (UID: "fbb268b8-644a-41c8-a78b-2779914d0e9d"). InnerVolumeSpecName "kube-api-access-4dg4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:54.389086 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.389031    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fbb268b8-644a-41c8-a78b-2779914d0e9d-maas-api-tls" pod "fbb268b8-644a-41c8-a78b-2779914d0e9d" (UID: "fbb268b8-644a-41c8-a78b-2779914d0e9d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:54.506195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.494406    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fbb268b8-644a-41c8-a78b-2779914d0e9d-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:54.506195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.494441    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fbb268b8-644a-41c8-a78b-2779914d0e9d-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:54.506195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.494456    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dg4v\" (UniqueName: \"kubernetes.io/projected/fbb268b8-644a-41c8-a78b-2779914d0e9d-kube-api-access-4dg4v\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:54.724247 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.721465    4524 generic.go:356] "Generic (PLEG): container finished" podID="fbb268b8-644a-41c8-a78b-2779914d0e9d" containerID="fdcf6fb1b03028e373db4d39d4a5f86ab67afd01659d86a44a9b934e65a6c131" exitCode=0
Oct 30 19:46:54.724247 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.721599    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr" event={"ID":"fbb268b8-644a-41c8-a78b-2779914d0e9d","Type":"ContainerDied","Data":"fdcf6fb1b03028e373db4d39d4a5f86ab67afd01659d86a44a9b934e65a6c131"}
Oct 30 19:46:54.724247 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.721648    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr" event={"ID":"fbb268b8-644a-41c8-a78b-2779914d0e9d","Type":"ContainerDied","Data":"3cbd6ce2cf6aa2885222397eabea6eae4e30d6f8cf6e31ad76deca35d60a3c90"}
Oct 30 19:46:54.724247 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.721677    4524 scope.go:122] "RemoveContainer" containerID="fdcf6fb1b03028e373db4d39d4a5f86ab67afd01659d86a44a9b934e65a6c131"
Oct 30 19:46:54.724247 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.721838    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"
Oct 30 19:46:54.781479 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.779662    4524 scope.go:122] "RemoveContainer" containerID="fdcf6fb1b03028e373db4d39d4a5f86ab67afd01659d86a44a9b934e65a6c131"
Oct 30 19:46:54.781479 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:46:54.780387    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdcf6fb1b03028e373db4d39d4a5f86ab67afd01659d86a44a9b934e65a6c131\": container with ID starting with fdcf6fb1b03028e373db4d39d4a5f86ab67afd01659d86a44a9b934e65a6c131 not found: ID does not exist" containerID="fdcf6fb1b03028e373db4d39d4a5f86ab67afd01659d86a44a9b934e65a6c131"
Oct 30 19:46:54.781479 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.780428    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdcf6fb1b03028e373db4d39d4a5f86ab67afd01659d86a44a9b934e65a6c131"} err="failed to get container status \"fdcf6fb1b03028e373db4d39d4a5f86ab67afd01659d86a44a9b934e65a6c131\": rpc error: code = NotFound desc = could not find container \"fdcf6fb1b03028e373db4d39d4a5f86ab67afd01659d86a44a9b934e65a6c131\": container with ID starting with fdcf6fb1b03028e373db4d39d4a5f86ab67afd01659d86a44a9b934e65a6c131 not found: ID does not exist"
Oct 30 19:46:54.789290 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.789255    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"]
Oct 30 19:46:54.850135 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:54.834668    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-cfa4bb-58b74bd585-zw2wr"]
Oct 30 19:46:55.746787 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:55.746695    4524 generic.go:356] "Generic (PLEG): container finished" podID="050d3ce5-caa8-4f50-8328-bef71892924b" containerID="73fae5e4e2af401846687690f9896c2b2158ba01c48bf89fdc5013b73e6f017d" exitCode=0
Oct 30 19:46:55.747149 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:55.746921    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s" event={"ID":"050d3ce5-caa8-4f50-8328-bef71892924b","Type":"ContainerDied","Data":"73fae5e4e2af401846687690f9896c2b2158ba01c48bf89fdc5013b73e6f017d"}
Oct 30 19:46:55.856250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:55.856192    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s"
Oct 30 19:46:55.924805 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:55.923306    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/050d3ce5-caa8-4f50-8328-bef71892924b-kube-api-access-g6skj\" (UniqueName: \"kubernetes.io/projected/050d3ce5-caa8-4f50-8328-bef71892924b-kube-api-access-g6skj\") pod \"050d3ce5-caa8-4f50-8328-bef71892924b\" (UID: \"050d3ce5-caa8-4f50-8328-bef71892924b\") "
Oct 30 19:46:55.924805 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:55.923416    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/050d3ce5-caa8-4f50-8328-bef71892924b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/050d3ce5-caa8-4f50-8328-bef71892924b-plugins-config-volume\") pod \"050d3ce5-caa8-4f50-8328-bef71892924b\" (UID: \"050d3ce5-caa8-4f50-8328-bef71892924b\") "
Oct 30 19:46:55.924805 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:55.924478    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/050d3ce5-caa8-4f50-8328-bef71892924b-plugins-config-volume" pod "050d3ce5-caa8-4f50-8328-bef71892924b" (UID: "050d3ce5-caa8-4f50-8328-bef71892924b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:55.937480 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:55.937449    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/050d3ce5-caa8-4f50-8328-bef71892924b-kube-api-access-g6skj" pod "050d3ce5-caa8-4f50-8328-bef71892924b" (UID: "050d3ce5-caa8-4f50-8328-bef71892924b"). InnerVolumeSpecName "kube-api-access-g6skj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:56.025352 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:56.025278    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g6skj\" (UniqueName: \"kubernetes.io/projected/050d3ce5-caa8-4f50-8328-bef71892924b-kube-api-access-g6skj\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:56.025352 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:56.025306    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/050d3ce5-caa8-4f50-8328-bef71892924b-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:56.460198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:56.457739    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fbb268b8-644a-41c8-a78b-2779914d0e9d" path="/var/lib/kubelet/pods/fbb268b8-644a-41c8-a78b-2779914d0e9d/volumes"
Oct 30 19:46:56.758899 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:56.758821    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s" event={"ID":"050d3ce5-caa8-4f50-8328-bef71892924b","Type":"ContainerDied","Data":"e2b2b46cee4ce9045ca78b399b826d46d8db0e9b4e0e532d799794240387f6ac"}
Oct 30 19:46:56.758899 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:56.758878    4524 scope.go:122] "RemoveContainer" containerID="73fae5e4e2af401846687690f9896c2b2158ba01c48bf89fdc5013b73e6f017d"
Oct 30 19:46:56.759448 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:56.758825    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s"
Oct 30 19:46:56.802219 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:56.801434    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s"]
Oct 30 19:46:56.806221 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:56.806191    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s"]
Oct 30 19:46:56.824335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:56.818359    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88"]
Oct 30 19:46:56.824335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:56.818754    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w0-1e7123-c96dc489b-6ck4s" podUID="050d3ce5-caa8-4f50-8328-bef71892924b" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.114:50052\" within 1s: context deadline exceeded"
Oct 30 19:46:56.824335 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:46:56.819462    4524 logging.go:55] [core] [Channel #7499 SubChannel #7500]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.114:50052", ServerName: "10.133.0.114:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.114:50052: operation was canceled"
Oct 30 19:46:58.204506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.202488    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"]
Oct 30 19:46:58.204506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.203250    4524 cpu_manager.go:414] "containerMap: removing container" podUID="050d3ce5-caa8-4f50-8328-bef71892924b" containerName="payload-processing"
Oct 30 19:46:58.204506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.203264    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="050d3ce5-caa8-4f50-8328-bef71892924b" containerName="payload-processing"
Oct 30 19:46:58.204506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.203314    4524 cpu_manager.go:414] "containerMap: removing container" podUID="fbb268b8-644a-41c8-a78b-2779914d0e9d" containerName="maas-api"
Oct 30 19:46:58.204506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.203319    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbb268b8-644a-41c8-a78b-2779914d0e9d" containerName="maas-api"
Oct 30 19:46:58.204506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.203425    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="fbb268b8-644a-41c8-a78b-2779914d0e9d" containerName="maas-api"
Oct 30 19:46:58.204506 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.203442    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="050d3ce5-caa8-4f50-8328-bef71892924b" containerName="payload-processing"
Oct 30 19:46:58.208842 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.208817    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.211817 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.211794    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"]
Oct 30 19:46:58.226507 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.226274    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0beff1a-564e-439e-a451-724039ee76a0-istiod-ca-cert\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.226507 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.226439    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-istio-envoy\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.226704 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.226548    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-workload-certs\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.226704 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.226611    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0beff1a-564e-439e-a451-724039ee76a0-istio-podinfo\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.226829 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.226774    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0beff1a-564e-439e-a451-724039ee76a0-istio-token\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.226829 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.226822    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-workload-socket\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.226946 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.226858    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-credential-socket\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.227007 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.226958    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8hxm\" (UniqueName: \"kubernetes.io/projected/b0beff1a-564e-439e-a451-724039ee76a0-kube-api-access-r8hxm\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.227068 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.227047    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-istio-data\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.347238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.344903    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-workload-certs\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.347238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.344948    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0beff1a-564e-439e-a451-724039ee76a0-istio-podinfo\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.347238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.344979    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0beff1a-564e-439e-a451-724039ee76a0-istio-token\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.347238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.344998    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-workload-socket\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.347238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.345019    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-credential-socket\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.347238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.345076    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8hxm\" (UniqueName: \"kubernetes.io/projected/b0beff1a-564e-439e-a451-724039ee76a0-kube-api-access-r8hxm\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.347238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.345128    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-istio-data\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.347238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.345236    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0beff1a-564e-439e-a451-724039ee76a0-istiod-ca-cert\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.347238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.345338    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-istio-envoy\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.347238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.346068    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-workload-socket\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.347238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.346653    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0beff1a-564e-439e-a451-724039ee76a0-istiod-ca-cert\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.347238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.346936    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-credential-socket\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.347238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.346949    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-workload-certs\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.347238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.347128    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-istio-data\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.349938 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.349915    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0beff1a-564e-439e-a451-724039ee76a0-istio-podinfo\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.353276 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.353255    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-istio-envoy\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.405570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.405536    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8hxm\" (UniqueName: \"kubernetes.io/projected/b0beff1a-564e-439e-a451-724039ee76a0-kube-api-access-r8hxm\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.406317 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.406263    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0beff1a-564e-439e-a451-724039ee76a0-istio-token\") pod \"e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") " pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.414156 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.414130    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="050d3ce5-caa8-4f50-8328-bef71892924b" path="/var/lib/kubelet/pods/050d3ce5-caa8-4f50-8328-bef71892924b/volumes"
Oct 30 19:46:58.538399 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.538320    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:46:58.708076 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.706853    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"
Oct 30 19:46:58.861253 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.861222    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"]
Oct 30 19:46:58.862658 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.862628    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:46:58.862716 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.862694    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:46:58.862753 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.862722    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:46:58.916238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.913759    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"]
Oct 30 19:46:58.916238 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:58.914004    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d" podUID="08eef1b9-f940-48f4-bb6d-b355616ded55" containerName="maas-api" containerID="cri-o://fc4b9846c54439357da91972c646fcfd66275ef88512b4c6dfc57456371da239" gracePeriod=30
Oct 30 19:46:59.257928 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:59.256303    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d" podUID="08eef1b9-f940-48f4-bb6d-b355616ded55" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.178:8443/health\": dial tcp 10.133.0.178:8443: connect: connection refused"
Oct 30 19:46:59.859395 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:59.859364    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"
Oct 30 19:46:59.871021 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:59.870990    4524 generic.go:356] "Generic (PLEG): container finished" podID="08eef1b9-f940-48f4-bb6d-b355616ded55" containerID="fc4b9846c54439357da91972c646fcfd66275ef88512b4c6dfc57456371da239" exitCode=0
Oct 30 19:46:59.871434 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:59.871408    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d" event={"ID":"08eef1b9-f940-48f4-bb6d-b355616ded55","Type":"ContainerDied","Data":"fc4b9846c54439357da91972c646fcfd66275ef88512b4c6dfc57456371da239"}
Oct 30 19:46:59.871623 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:59.871606    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d" event={"ID":"08eef1b9-f940-48f4-bb6d-b355616ded55","Type":"ContainerDied","Data":"2b4b362ef8e691306bf1d228c225fd586766297b5e20158622b53f3fcaed4744"}
Oct 30 19:46:59.871738 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:59.871726    4524 scope.go:122] "RemoveContainer" containerID="fc4b9846c54439357da91972c646fcfd66275ef88512b4c6dfc57456371da239"
Oct 30 19:46:59.871985 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:59.871968    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"
Oct 30 19:46:59.876314 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:59.876290    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz" event={"ID":"b0beff1a-564e-439e-a451-724039ee76a0","Type":"ContainerStarted","Data":"62d17857ed1a55f61b9226a3652fc87153c5457bb7b6dbee9070e3d0085f061a"}
Oct 30 19:46:59.876504 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:59.876484    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz" event={"ID":"b0beff1a-564e-439e-a451-724039ee76a0","Type":"ContainerStarted","Data":"f7d8b29c45d29dc24beb44e359931c01d2e99a49f5a6022039bede981af4fe3d"}
Oct 30 19:46:59.905213 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:59.889088    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/08eef1b9-f940-48f4-bb6d-b355616ded55-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08eef1b9-f940-48f4-bb6d-b355616ded55-openshift-service-ca\") pod \"08eef1b9-f940-48f4-bb6d-b355616ded55\" (UID: \"08eef1b9-f940-48f4-bb6d-b355616ded55\") "
Oct 30 19:46:59.905533 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:59.905509    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/08eef1b9-f940-48f4-bb6d-b355616ded55-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08eef1b9-f940-48f4-bb6d-b355616ded55-maas-api-tls\") pod \"08eef1b9-f940-48f4-bb6d-b355616ded55\" (UID: \"08eef1b9-f940-48f4-bb6d-b355616ded55\") "
Oct 30 19:46:59.905750 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:59.905727    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/08eef1b9-f940-48f4-bb6d-b355616ded55-kube-api-access-wzlqs\" (UniqueName: \"kubernetes.io/projected/08eef1b9-f940-48f4-bb6d-b355616ded55-kube-api-access-wzlqs\") pod \"08eef1b9-f940-48f4-bb6d-b355616ded55\" (UID: \"08eef1b9-f940-48f4-bb6d-b355616ded55\") "
Oct 30 19:46:59.905977 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:59.905959    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/08eef1b9-f940-48f4-bb6d-b355616ded55-metrics-certs\" (UniqueName: \"kubernetes.io/secret/08eef1b9-f940-48f4-bb6d-b355616ded55-metrics-certs\") pod \"08eef1b9-f940-48f4-bb6d-b355616ded55\" (UID: \"08eef1b9-f940-48f4-bb6d-b355616ded55\") "
Oct 30 19:46:59.939441 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:59.937321    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08eef1b9-f940-48f4-bb6d-b355616ded55-metrics-certs" pod "08eef1b9-f940-48f4-bb6d-b355616ded55" (UID: "08eef1b9-f940-48f4-bb6d-b355616ded55"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:59.939441 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:59.937785    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08eef1b9-f940-48f4-bb6d-b355616ded55-openshift-service-ca" pod "08eef1b9-f940-48f4-bb6d-b355616ded55" (UID: "08eef1b9-f940-48f4-bb6d-b355616ded55"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:59.942363 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:59.942295    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08eef1b9-f940-48f4-bb6d-b355616ded55-kube-api-access-wzlqs" pod "08eef1b9-f940-48f4-bb6d-b355616ded55" (UID: "08eef1b9-f940-48f4-bb6d-b355616ded55"). InnerVolumeSpecName "kube-api-access-wzlqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:59.944247 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:59.944220    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08eef1b9-f940-48f4-bb6d-b355616ded55-maas-api-tls" pod "08eef1b9-f940-48f4-bb6d-b355616ded55" (UID: "08eef1b9-f940-48f4-bb6d-b355616ded55"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:46:59.963612 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:59.963544    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz" podStartSLOduration=1.963524872 podStartE2EDuration="1.963524872s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:46:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:46:59.95909191 +0000 UTC m=+2451.700082989" watchObservedRunningTime="2026-10-30 19:46:59.963524872 +0000 UTC m=+2451.704515952"
Oct 30 19:46:59.967671 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:59.967652    4524 scope.go:122] "RemoveContainer" containerID="fc4b9846c54439357da91972c646fcfd66275ef88512b4c6dfc57456371da239"
Oct 30 19:46:59.968693 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:46:59.968664    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc4b9846c54439357da91972c646fcfd66275ef88512b4c6dfc57456371da239\": container with ID starting with fc4b9846c54439357da91972c646fcfd66275ef88512b4c6dfc57456371da239 not found: ID does not exist" containerID="fc4b9846c54439357da91972c646fcfd66275ef88512b4c6dfc57456371da239"
Oct 30 19:46:59.968854 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:46:59.968827    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc4b9846c54439357da91972c646fcfd66275ef88512b4c6dfc57456371da239"} err="failed to get container status \"fc4b9846c54439357da91972c646fcfd66275ef88512b4c6dfc57456371da239\": rpc error: code = NotFound desc = could not find container \"fc4b9846c54439357da91972c646fcfd66275ef88512b4c6dfc57456371da239\": container with ID starting with fc4b9846c54439357da91972c646fcfd66275ef88512b4c6dfc57456371da239 not found: ID does not exist"
Oct 30 19:47:00.009552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:00.009516    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/08eef1b9-f940-48f4-bb6d-b355616ded55-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:00.009552 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:00.009554    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/08eef1b9-f940-48f4-bb6d-b355616ded55-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:00.009746 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:00.009572    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzlqs\" (UniqueName: \"kubernetes.io/projected/08eef1b9-f940-48f4-bb6d-b355616ded55-kube-api-access-wzlqs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:00.009746 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:00.009589    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/08eef1b9-f940-48f4-bb6d-b355616ded55-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:00.260769 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:00.260745    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"]
Oct 30 19:47:00.267069 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:00.267042    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1a844e-5778b45bff-48h6d"]
Oct 30 19:47:00.421826 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:00.421786    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="08eef1b9-f940-48f4-bb6d-b355616ded55" path="/var/lib/kubelet/pods/08eef1b9-f940-48f4-bb6d-b355616ded55/volumes"
Oct 30 19:47:00.539798 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:00.539759    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:47:00.540386 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:00.540356    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused" start-of-body=
Oct 30 19:47:00.540527 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:00.540410    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz" podUID="b0beff1a-564e-439e-a451-724039ee76a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused"
Oct 30 19:47:01.539161 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:01.539118    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused" start-of-body=
Oct 30 19:47:01.539614 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:01.539200    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz" podUID="b0beff1a-564e-439e-a451-724039ee76a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused"
Oct 30 19:47:01.819350 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:01.819239    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88" podUID="0806e982-fe3c-496d-95e2-c75d2b23dacf" containerName="payload-processing" containerID="cri-o://aa01e1114c8827bc3a74044e3300d71e54db7d06bdf336f5daa00afa3db99880" gracePeriod=25
Oct 30 19:47:02.539582 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:02.539547    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused" start-of-body=
Oct 30 19:47:02.540041 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:02.539599    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz" podUID="b0beff1a-564e-439e-a451-724039ee76a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused"
Oct 30 19:47:02.915647 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:02.915609    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"]
Oct 30 19:47:02.916188 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:02.916128    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp" podUID="e6dd71b4-1741-4da3-99f4-82121e585d89" containerName="istio-proxy" containerID="cri-o://c37d23bb716ffaa764e13750ec186b6e3c3d43cdcd4bd6b8e493f90844e96c8c" gracePeriod=30
Oct 30 19:47:03.511764 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:03.511723    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88" podUID="0806e982-fe3c-496d-95e2-c75d2b23dacf" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:47:03.538848 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:03.538811    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused" start-of-body=
Oct 30 19:47:03.539042 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:03.538871    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz" podUID="b0beff1a-564e-439e-a451-724039ee76a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused"
Oct 30 19:47:04.546614 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:04.539349    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused" start-of-body=
Oct 30 19:47:04.546614 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:04.539407    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz" podUID="b0beff1a-564e-439e-a451-724039ee76a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused"
Oct 30 19:47:05.560255 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:05.546606    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused" start-of-body=
Oct 30 19:47:05.560255 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:05.546710    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz" podUID="b0beff1a-564e-439e-a451-724039ee76a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.188:15021/healthz/ready\": dial tcp 10.133.0.188:15021: connect: connection refused"
Oct 30 19:47:06.285674 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:06.285630    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.183:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:47:06.285997 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:06.285960    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km" podUID="0d49fd92-802a-402c-8098-414925be3659" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.183:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:47:06.547652 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:06.547569    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:47:06.549194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:06.549150    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:47:06.552815 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:06.552794    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:47:07.757316 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:07.757222    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"]
Oct 30 19:47:07.758304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:07.757474    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79" podUID="5c074845-90e7-46eb-bed0-9360969cf36e" containerName="istio-proxy" containerID="cri-o://2d20b11e092e9e4fa2d099d2c88e2233190cad23d1cf576c7e8ec8bd4e2ccdad" gracePeriod=30
Oct 30 19:47:07.789778 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:07.789738    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"]
Oct 30 19:47:07.796319 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:07.796285    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"]
Oct 30 19:47:07.796558 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:07.796533    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2" podUID="0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" containerName="istio-proxy" containerID="cri-o://a0c7c04ab829c65b0dc75c9b6908e40ab662c1d8d88481a8edf1e6e684138c28" gracePeriod=30
Oct 30 19:47:08.129479 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.129441    4524 generic.go:356] "Generic (PLEG): container finished" podID="0d49fd92-802a-402c-8098-414925be3659" containerID="cb9a95485bf80337be1c2764e89bc5f1000c889a2517d4072846ecd741ba4595" exitCode=0
Oct 30 19:47:08.129658 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.129530    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km" event={"ID":"0d49fd92-802a-402c-8098-414925be3659","Type":"ContainerDied","Data":"cb9a95485bf80337be1c2764e89bc5f1000c889a2517d4072846ecd741ba4595"}
Oct 30 19:47:08.131399 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.131376    4524 generic.go:356] "Generic (PLEG): container finished" podID="e6dd71b4-1741-4da3-99f4-82121e585d89" containerID="c37d23bb716ffaa764e13750ec186b6e3c3d43cdcd4bd6b8e493f90844e96c8c" exitCode=0
Oct 30 19:47:08.131529 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.131442    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp" event={"ID":"e6dd71b4-1741-4da3-99f4-82121e585d89","Type":"ContainerDied","Data":"c37d23bb716ffaa764e13750ec186b6e3c3d43cdcd4bd6b8e493f90844e96c8c"}
Oct 30 19:47:08.131675 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.131659    4524 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz" secret="" err="secret \"e2e-ipp-b-1a844e-openshift-default-dockercfg-465qc\" not found"
Oct 30 19:47:08.132228 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.132210    4524 scope.go:122] "RemoveContainer" containerID="cb9a95485bf80337be1c2764e89bc5f1000c889a2517d4072846ecd741ba4595"
Oct 30 19:47:08.134160 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.134001    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:47:08.134160 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.134088    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:47:08.134160 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.134137    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:47:08.291087 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.291052    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:47:08.427689 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.427658    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6dd71b4-1741-4da3-99f4-82121e585d89-kube-api-access-rm6fb\" (UniqueName: \"kubernetes.io/projected/e6dd71b4-1741-4da3-99f4-82121e585d89-kube-api-access-rm6fb\") pod \"e6dd71b4-1741-4da3-99f4-82121e585d89\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") "
Oct 30 19:47:08.427880 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.427712    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-token\" (UniqueName: \"kubernetes.io/projected/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-token\") pod \"e6dd71b4-1741-4da3-99f4-82121e585d89\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") "
Oct 30 19:47:08.427880 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.427768    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-workload-socket\") pod \"e6dd71b4-1741-4da3-99f4-82121e585d89\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") "
Oct 30 19:47:08.427880 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.427809    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-workload-certs\") pod \"e6dd71b4-1741-4da3-99f4-82121e585d89\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") "
Oct 30 19:47:08.427880 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.427837    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-data\") pod \"e6dd71b4-1741-4da3-99f4-82121e585d89\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") "
Oct 30 19:47:08.427880 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.427876    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e6dd71b4-1741-4da3-99f4-82121e585d89-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6dd71b4-1741-4da3-99f4-82121e585d89-istiod-ca-cert\") pod \"e6dd71b4-1741-4da3-99f4-82121e585d89\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") "
Oct 30 19:47:08.428404 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.427969    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-podinfo\") pod \"e6dd71b4-1741-4da3-99f4-82121e585d89\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") "
Oct 30 19:47:08.428404 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.428040    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-envoy\") pod \"e6dd71b4-1741-4da3-99f4-82121e585d89\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") "
Oct 30 19:47:08.428404 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.428093    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-credential-socket\") pod \"e6dd71b4-1741-4da3-99f4-82121e585d89\" (UID: \"e6dd71b4-1741-4da3-99f4-82121e585d89\") "
Oct 30 19:47:08.428404 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.428223    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-workload-certs" pod "e6dd71b4-1741-4da3-99f4-82121e585d89" (UID: "e6dd71b4-1741-4da3-99f4-82121e585d89"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:08.428840 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.428813    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:08.429032 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.429008    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-credential-socket" pod "e6dd71b4-1741-4da3-99f4-82121e585d89" (UID: "e6dd71b4-1741-4da3-99f4-82121e585d89"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:08.429467 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.429441    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-data" pod "e6dd71b4-1741-4da3-99f4-82121e585d89" (UID: "e6dd71b4-1741-4da3-99f4-82121e585d89"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:08.429623 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.429601    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6dd71b4-1741-4da3-99f4-82121e585d89-istiod-ca-cert" pod "e6dd71b4-1741-4da3-99f4-82121e585d89" (UID: "e6dd71b4-1741-4da3-99f4-82121e585d89"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:08.429907 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.429883    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-workload-socket" pod "e6dd71b4-1741-4da3-99f4-82121e585d89" (UID: "e6dd71b4-1741-4da3-99f4-82121e585d89"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:08.431598 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.431568    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6dd71b4-1741-4da3-99f4-82121e585d89-kube-api-access-rm6fb" pod "e6dd71b4-1741-4da3-99f4-82121e585d89" (UID: "e6dd71b4-1741-4da3-99f4-82121e585d89"). InnerVolumeSpecName "kube-api-access-rm6fb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:08.432002 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.431944    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-podinfo" pod "e6dd71b4-1741-4da3-99f4-82121e585d89" (UID: "e6dd71b4-1741-4da3-99f4-82121e585d89"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:47:08.432094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.432074    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-token" pod "e6dd71b4-1741-4da3-99f4-82121e585d89" (UID: "e6dd71b4-1741-4da3-99f4-82121e585d89"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:08.435521 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.435494    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-envoy" pod "e6dd71b4-1741-4da3-99f4-82121e585d89" (UID: "e6dd71b4-1741-4da3-99f4-82121e585d89"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:08.530185 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.530139    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rm6fb\" (UniqueName: \"kubernetes.io/projected/e6dd71b4-1741-4da3-99f4-82121e585d89-kube-api-access-rm6fb\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:08.530185 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.530182    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:08.530185 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.530193    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:08.530452 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.530202    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:08.530452 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.530210    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6dd71b4-1741-4da3-99f4-82121e585d89-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:08.530452 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.530220    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:08.530452 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.530228    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:08.530452 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:08.530237    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6dd71b4-1741-4da3-99f4-82121e585d89-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:09.140770 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:09.140739    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"
Oct 30 19:47:09.140770 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:09.140763    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp" event={"ID":"e6dd71b4-1741-4da3-99f4-82121e585d89","Type":"ContainerDied","Data":"f7d1f7a4a23f1f364b9c1fed6310a7f8eafc7dacc2565457f7cc24a18cf04c7b"}
Oct 30 19:47:09.141518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:09.140839    4524 scope.go:122] "RemoveContainer" containerID="c37d23bb716ffaa764e13750ec186b6e3c3d43cdcd4bd6b8e493f90844e96c8c"
Oct 30 19:47:09.143742 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:09.143714    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz" podUID="b0beff1a-564e-439e-a451-724039ee76a0" containerName="istio-proxy" containerID="cri-o://62d17857ed1a55f61b9226a3652fc87153c5457bb7b6dbee9070e3d0085f061a" gracePeriod=30
Oct 30 19:47:09.143993 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:09.143971    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km" event={"ID":"0d49fd92-802a-402c-8098-414925be3659","Type":"ContainerStarted","Data":"37e2a6fa7b944865e186f2fa269fcf61a73a27f009c58cd30faaab6a2b52395e"}
Oct 30 19:47:09.226312 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:09.226282    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"]
Oct 30 19:47:09.242759 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:09.242718    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-cfa4bb-openshift-default-75f585bdc-ngbfp"]
Oct 30 19:47:09.507600 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:09.506238    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:47:09.507600 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:09.506386    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused" start-of-body=
Oct 30 19:47:09.507600 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:09.506437    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km" podUID="0d49fd92-802a-402c-8098-414925be3659" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused"
Oct 30 19:47:10.492193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:10.457673    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e6dd71b4-1741-4da3-99f4-82121e585d89" path="/var/lib/kubelet/pods/e6dd71b4-1741-4da3-99f4-82121e585d89/volumes"
Oct 30 19:47:10.543235 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:10.507537    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused" start-of-body=
Oct 30 19:47:10.543589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:10.543550    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km" podUID="0d49fd92-802a-402c-8098-414925be3659" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused"
Oct 30 19:47:11.506476 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:11.506439    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused" start-of-body=
Oct 30 19:47:11.506971 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:11.506942    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km" podUID="0d49fd92-802a-402c-8098-414925be3659" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused"
Oct 30 19:47:12.507036 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:12.506967    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused" start-of-body=
Oct 30 19:47:12.507036 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:12.507024    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km" podUID="0d49fd92-802a-402c-8098-414925be3659" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused"
Oct 30 19:47:13.227457 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.227397    4524 generic.go:356] "Generic (PLEG): container finished" podID="5c074845-90e7-46eb-bed0-9360969cf36e" containerID="2d20b11e092e9e4fa2d099d2c88e2233190cad23d1cf576c7e8ec8bd4e2ccdad" exitCode=0
Oct 30 19:47:13.228060 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.227766    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79" event={"ID":"5c074845-90e7-46eb-bed0-9360969cf36e","Type":"ContainerDied","Data":"2d20b11e092e9e4fa2d099d2c88e2233190cad23d1cf576c7e8ec8bd4e2ccdad"}
Oct 30 19:47:13.230869 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.230202    4524 generic.go:356] "Generic (PLEG): container finished" podID="0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" containerID="a0c7c04ab829c65b0dc75c9b6908e40ab662c1d8d88481a8edf1e6e684138c28" exitCode=0
Oct 30 19:47:13.230869 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.230246    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2" event={"ID":"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457","Type":"ContainerDied","Data":"a0c7c04ab829c65b0dc75c9b6908e40ab662c1d8d88481a8edf1e6e684138c28"}
Oct 30 19:47:13.325216 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.324797    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:47:13.425748 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.425413    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:47:13.506598 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.506559    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused" start-of-body=
Oct 30 19:47:13.506793 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.506616    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km" podUID="0d49fd92-802a-402c-8098-414925be3659" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused"
Oct 30 19:47:13.513090 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.513044    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88" podUID="0806e982-fe3c-496d-95e2-c75d2b23dacf" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:47:13.516531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.515966    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-istio-data\") pod \"5c074845-90e7-46eb-bed0-9360969cf36e\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") "
Oct 30 19:47:13.516531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.516085    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c074845-90e7-46eb-bed0-9360969cf36e-kube-api-access-v67jp\" (UniqueName: \"kubernetes.io/projected/5c074845-90e7-46eb-bed0-9360969cf36e-kube-api-access-v67jp\") pod \"5c074845-90e7-46eb-bed0-9360969cf36e\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") "
Oct 30 19:47:13.516531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.516106    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c074845-90e7-46eb-bed0-9360969cf36e-istio-token\" (UniqueName: \"kubernetes.io/projected/5c074845-90e7-46eb-bed0-9360969cf36e-istio-token\") pod \"5c074845-90e7-46eb-bed0-9360969cf36e\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") "
Oct 30 19:47:13.516531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.516161    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-workload-certs\") pod \"5c074845-90e7-46eb-bed0-9360969cf36e\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") "
Oct 30 19:47:13.516531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.516201    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5c074845-90e7-46eb-bed0-9360969cf36e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c074845-90e7-46eb-bed0-9360969cf36e-istio-podinfo\") pod \"5c074845-90e7-46eb-bed0-9360969cf36e\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") "
Oct 30 19:47:13.516531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.516224    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c074845-90e7-46eb-bed0-9360969cf36e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c074845-90e7-46eb-bed0-9360969cf36e-istiod-ca-cert\") pod \"5c074845-90e7-46eb-bed0-9360969cf36e\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") "
Oct 30 19:47:13.516531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.516243    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-istio-envoy\") pod \"5c074845-90e7-46eb-bed0-9360969cf36e\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") "
Oct 30 19:47:13.516531 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.516361    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-credential-socket\") pod \"5c074845-90e7-46eb-bed0-9360969cf36e\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") "
Oct 30 19:47:13.516891 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.516627    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-credential-socket" pod "5c074845-90e7-46eb-bed0-9360969cf36e" (UID: "5c074845-90e7-46eb-bed0-9360969cf36e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:13.517366 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.517345    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-podinfo\") pod \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") "
Oct 30 19:47:13.517444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.517381    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-workload-socket\") pod \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") "
Oct 30 19:47:13.517444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.517403    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-data\") pod \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") "
Oct 30 19:47:13.517444 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.517435    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-workload-socket\") pod \"5c074845-90e7-46eb-bed0-9360969cf36e\" (UID: \"5c074845-90e7-46eb-bed0-9360969cf36e\") "
Oct 30 19:47:13.517544 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.517507    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-envoy\") pod \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") "
Oct 30 19:47:13.517544 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.517529    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-credential-socket\") pod \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") "
Oct 30 19:47:13.518040 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.518026    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:13.519889 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.518977    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-credential-socket" pod "0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" (UID: "0be49b2f-fdee-4ba1-9e18-3b1edcb6a457"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:13.519889 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.519335    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-istio-envoy" pod "5c074845-90e7-46eb-bed0-9360969cf36e" (UID: "5c074845-90e7-46eb-bed0-9360969cf36e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:13.519889 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.519497    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-workload-socket" pod "0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" (UID: "0be49b2f-fdee-4ba1-9e18-3b1edcb6a457"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:13.519889 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.519638    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-data" pod "0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" (UID: "0be49b2f-fdee-4ba1-9e18-3b1edcb6a457"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:13.519889 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.519657    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-workload-certs" pod "5c074845-90e7-46eb-bed0-9360969cf36e" (UID: "5c074845-90e7-46eb-bed0-9360969cf36e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:13.519889 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.519770    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-istio-data" pod "5c074845-90e7-46eb-bed0-9360969cf36e" (UID: "5c074845-90e7-46eb-bed0-9360969cf36e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:13.519889 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.519813    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-workload-socket" pod "5c074845-90e7-46eb-bed0-9360969cf36e" (UID: "5c074845-90e7-46eb-bed0-9360969cf36e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:13.520469 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.520255    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c074845-90e7-46eb-bed0-9360969cf36e-istiod-ca-cert" pod "5c074845-90e7-46eb-bed0-9360969cf36e" (UID: "5c074845-90e7-46eb-bed0-9360969cf36e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:13.521990 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.521858    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-envoy" pod "0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" (UID: "0be49b2f-fdee-4ba1-9e18-3b1edcb6a457"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:13.521990 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.521875    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c074845-90e7-46eb-bed0-9360969cf36e-istio-podinfo" pod "5c074845-90e7-46eb-bed0-9360969cf36e" (UID: "5c074845-90e7-46eb-bed0-9360969cf36e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:47:13.521990 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.521962    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c074845-90e7-46eb-bed0-9360969cf36e-istio-token" pod "5c074845-90e7-46eb-bed0-9360969cf36e" (UID: "5c074845-90e7-46eb-bed0-9360969cf36e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:13.522916 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.522894    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c074845-90e7-46eb-bed0-9360969cf36e-kube-api-access-v67jp" pod "5c074845-90e7-46eb-bed0-9360969cf36e" (UID: "5c074845-90e7-46eb-bed0-9360969cf36e"). InnerVolumeSpecName "kube-api-access-v67jp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:13.526562 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.526529    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-podinfo" pod "0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" (UID: "0be49b2f-fdee-4ba1-9e18-3b1edcb6a457"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:47:13.629196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.628696    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-token\" (UniqueName: \"kubernetes.io/projected/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-token\") pod \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") "
Oct 30 19:47:13.629196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.628810    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istiod-ca-cert\") pod \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") "
Oct 30 19:47:13.629196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.628859    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-kube-api-access-2775b\" (UniqueName: \"kubernetes.io/projected/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-kube-api-access-2775b\") pod \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") "
Oct 30 19:47:13.629196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.629089    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-workload-certs\") pod \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\" (UID: \"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457\") "
Oct 30 19:47:13.639215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.629894    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:13.639215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.629925    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:13.639215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.629944    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:13.639215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.629960    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v67jp\" (UniqueName: \"kubernetes.io/projected/5c074845-90e7-46eb-bed0-9360969cf36e-kube-api-access-v67jp\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:13.639215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.629977    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c074845-90e7-46eb-bed0-9360969cf36e-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:13.639215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.629992    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:13.639215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.630006    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c074845-90e7-46eb-bed0-9360969cf36e-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:13.639215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.630020    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c074845-90e7-46eb-bed0-9360969cf36e-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:13.639215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.630035    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:13.639215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.630049    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:13.639215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.630066    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:13.639215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.630078    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:13.639215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.630091    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c074845-90e7-46eb-bed0-9360969cf36e-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:13.639215 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.636683    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istiod-ca-cert" pod "0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" (UID: "0be49b2f-fdee-4ba1-9e18-3b1edcb6a457"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:13.660201 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.658579    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-token" pod "0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" (UID: "0be49b2f-fdee-4ba1-9e18-3b1edcb6a457"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:13.660201 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.659724    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-kube-api-access-2775b" pod "0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" (UID: "0be49b2f-fdee-4ba1-9e18-3b1edcb6a457"). InnerVolumeSpecName "kube-api-access-2775b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:13.660544 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.660515    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-workload-certs" pod "0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" (UID: "0be49b2f-fdee-4ba1-9e18-3b1edcb6a457"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:13.740194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.739895    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:13.740194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.739933    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:13.740194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.739951    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2775b\" (UniqueName: \"kubernetes.io/projected/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-kube-api-access-2775b\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:13.740194 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:13.739965    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:14.239389 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.239361    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"
Oct 30 19:47:14.239562 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.239367    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2" event={"ID":"0be49b2f-fdee-4ba1-9e18-3b1edcb6a457","Type":"ContainerDied","Data":"28093e303c74631ea03bd8af83c101bbb5f20314ea77a87f93a1caad51a386f0"}
Oct 30 19:47:14.239562 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.239538    4524 scope.go:122] "RemoveContainer" containerID="a0c7c04ab829c65b0dc75c9b6908e40ab662c1d8d88481a8edf1e6e684138c28"
Oct 30 19:47:14.241075 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.241059    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"
Oct 30 19:47:14.241198 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.241081    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79" event={"ID":"5c074845-90e7-46eb-bed0-9360969cf36e","Type":"ContainerDied","Data":"c016f2624d85db53edbd30f7830f88fa1438e5c6bff00c5f66794d282e125226"}
Oct 30 19:47:14.258239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.258220    4524 scope.go:122] "RemoveContainer" containerID="2d20b11e092e9e4fa2d099d2c88e2233190cad23d1cf576c7e8ec8bd4e2ccdad"
Oct 30 19:47:14.318459 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.318423    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"]
Oct 30 19:47:14.322926 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.322895    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-f5z79"]
Oct 30 19:47:14.333673 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.333475    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"]
Oct 30 19:47:14.338929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.338904    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-hdsr2"]
Oct 30 19:47:14.424212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.422253    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" path="/var/lib/kubelet/pods/0be49b2f-fdee-4ba1-9e18-3b1edcb6a457/volumes"
Oct 30 19:47:14.424212 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.423524    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c074845-90e7-46eb-bed0-9360969cf36e" path="/var/lib/kubelet/pods/5c074845-90e7-46eb-bed0-9360969cf36e/volumes"
Oct 30 19:47:14.508522 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.506441    4524 patch_prober.go:29] interesting pod/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused" start-of-body=
Oct 30 19:47:14.508522 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.506498    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km" podUID="0d49fd92-802a-402c-8098-414925be3659" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.183:15021/healthz/ready\": dial tcp 10.133.0.183:15021: connect: connection refused"
Oct 30 19:47:14.543946 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.543921    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:47:14.655761 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.655729    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b0beff1a-564e-439e-a451-724039ee76a0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0beff1a-564e-439e-a451-724039ee76a0-istiod-ca-cert\") pod \"b0beff1a-564e-439e-a451-724039ee76a0\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") "
Oct 30 19:47:14.655969 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.655926    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0beff1a-564e-439e-a451-724039ee76a0-kube-api-access-r8hxm\" (UniqueName: \"kubernetes.io/projected/b0beff1a-564e-439e-a451-724039ee76a0-kube-api-access-r8hxm\") pod \"b0beff1a-564e-439e-a451-724039ee76a0\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") "
Oct 30 19:47:14.656048 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.655976    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-istio-data\") pod \"b0beff1a-564e-439e-a451-724039ee76a0\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") "
Oct 30 19:47:14.656048 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.656016    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-workload-socket\") pod \"b0beff1a-564e-439e-a451-724039ee76a0\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") "
Oct 30 19:47:14.656048 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.656046    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-credential-socket\") pod \"b0beff1a-564e-439e-a451-724039ee76a0\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") "
Oct 30 19:47:14.656249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.656071    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0beff1a-564e-439e-a451-724039ee76a0-istio-token\" (UniqueName: \"kubernetes.io/projected/b0beff1a-564e-439e-a451-724039ee76a0-istio-token\") pod \"b0beff1a-564e-439e-a451-724039ee76a0\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") "
Oct 30 19:47:14.656249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.656211    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-istio-envoy\") pod \"b0beff1a-564e-439e-a451-724039ee76a0\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") "
Oct 30 19:47:14.656249 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.656214    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0beff1a-564e-439e-a451-724039ee76a0-istiod-ca-cert" pod "b0beff1a-564e-439e-a451-724039ee76a0" (UID: "b0beff1a-564e-439e-a451-724039ee76a0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:14.656378 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.656285    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b0beff1a-564e-439e-a451-724039ee76a0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0beff1a-564e-439e-a451-724039ee76a0-istio-podinfo\") pod \"b0beff1a-564e-439e-a451-724039ee76a0\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") "
Oct 30 19:47:14.656465 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.656439    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-workload-socket" pod "b0beff1a-564e-439e-a451-724039ee76a0" (UID: "b0beff1a-564e-439e-a451-724039ee76a0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:14.656717 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.656696    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-istio-data" pod "b0beff1a-564e-439e-a451-724039ee76a0" (UID: "b0beff1a-564e-439e-a451-724039ee76a0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:14.656805 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.656715    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-credential-socket" pod "b0beff1a-564e-439e-a451-724039ee76a0" (UID: "b0beff1a-564e-439e-a451-724039ee76a0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:14.656886 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.656867    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-workload-certs\") pod \"b0beff1a-564e-439e-a451-724039ee76a0\" (UID: \"b0beff1a-564e-439e-a451-724039ee76a0\") "
Oct 30 19:47:14.657123 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.657092    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-workload-certs" pod "b0beff1a-564e-439e-a451-724039ee76a0" (UID: "b0beff1a-564e-439e-a451-724039ee76a0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:14.657577 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.657554    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:14.657655 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.657583    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:14.657655 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.657601    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:14.657655 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.657616    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:14.657655 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.657630    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0beff1a-564e-439e-a451-724039ee76a0-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:14.658772 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.658742    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-istio-envoy" pod "b0beff1a-564e-439e-a451-724039ee76a0" (UID: "b0beff1a-564e-439e-a451-724039ee76a0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:14.660183 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.660143    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b0beff1a-564e-439e-a451-724039ee76a0-istio-podinfo" pod "b0beff1a-564e-439e-a451-724039ee76a0" (UID: "b0beff1a-564e-439e-a451-724039ee76a0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:47:14.660357 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.660332    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0beff1a-564e-439e-a451-724039ee76a0-istio-token" pod "b0beff1a-564e-439e-a451-724039ee76a0" (UID: "b0beff1a-564e-439e-a451-724039ee76a0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:14.660887 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.660863    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0beff1a-564e-439e-a451-724039ee76a0-kube-api-access-r8hxm" pod "b0beff1a-564e-439e-a451-724039ee76a0" (UID: "b0beff1a-564e-439e-a451-724039ee76a0"). InnerVolumeSpecName "kube-api-access-r8hxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:14.759407 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.759317    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0beff1a-564e-439e-a451-724039ee76a0-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:14.759407 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.759352    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0beff1a-564e-439e-a451-724039ee76a0-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:14.759407 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.759362    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8hxm\" (UniqueName: \"kubernetes.io/projected/b0beff1a-564e-439e-a451-724039ee76a0-kube-api-access-r8hxm\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:14.759407 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:14.759372    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0beff1a-564e-439e-a451-724039ee76a0-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:15.254876 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:15.254835    4524 generic.go:356] "Generic (PLEG): container finished" podID="b0beff1a-564e-439e-a451-724039ee76a0" containerID="62d17857ed1a55f61b9226a3652fc87153c5457bb7b6dbee9070e3d0085f061a" exitCode=0
Oct 30 19:47:15.254876 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:15.254878    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"
Oct 30 19:47:15.255114 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:15.254879    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz" event={"ID":"b0beff1a-564e-439e-a451-724039ee76a0","Type":"ContainerDied","Data":"62d17857ed1a55f61b9226a3652fc87153c5457bb7b6dbee9070e3d0085f061a"}
Oct 30 19:47:15.255114 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:15.254964    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz" event={"ID":"b0beff1a-564e-439e-a451-724039ee76a0","Type":"ContainerDied","Data":"f7d8b29c45d29dc24beb44e359931c01d2e99a49f5a6022039bede981af4fe3d"}
Oct 30 19:47:15.255114 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:15.254995    4524 scope.go:122] "RemoveContainer" containerID="62d17857ed1a55f61b9226a3652fc87153c5457bb7b6dbee9070e3d0085f061a"
Oct 30 19:47:15.268134 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:15.268117    4524 scope.go:122] "RemoveContainer" containerID="62d17857ed1a55f61b9226a3652fc87153c5457bb7b6dbee9070e3d0085f061a"
Oct 30 19:47:15.268417 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:47:15.268400    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62d17857ed1a55f61b9226a3652fc87153c5457bb7b6dbee9070e3d0085f061a\": container with ID starting with 62d17857ed1a55f61b9226a3652fc87153c5457bb7b6dbee9070e3d0085f061a not found: ID does not exist" containerID="62d17857ed1a55f61b9226a3652fc87153c5457bb7b6dbee9070e3d0085f061a"
Oct 30 19:47:15.268490 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:15.268428    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62d17857ed1a55f61b9226a3652fc87153c5457bb7b6dbee9070e3d0085f061a"} err="failed to get container status \"62d17857ed1a55f61b9226a3652fc87153c5457bb7b6dbee9070e3d0085f061a\": rpc error: code = NotFound desc = could not find container \"62d17857ed1a55f61b9226a3652fc87153c5457bb7b6dbee9070e3d0085f061a\": container with ID starting with 62d17857ed1a55f61b9226a3652fc87153c5457bb7b6dbee9070e3d0085f061a not found: ID does not exist"
Oct 30 19:47:15.284989 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:15.284943    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"]
Oct 30 19:47:15.288867 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:15.288839    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-1a844e-openshift-default-bcd6f4cc9-vt4dz"]
Oct 30 19:47:15.511036 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:15.510958    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:47:15.512359 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:15.512162    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:47:15.512504 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:15.512408    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:47:15.647414 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:47:15.647363    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:47:16.396622 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:16.396587    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b0beff1a-564e-439e-a451-724039ee76a0" path="/var/lib/kubelet/pods/b0beff1a-564e-439e-a451-724039ee76a0/volumes"
Oct 30 19:47:18.444201 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:18.442496    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64"]
Oct 30 19:47:22.118044 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:22.118011    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88"
Oct 30 19:47:22.237224 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:22.237188    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0806e982-fe3c-496d-95e2-c75d2b23dacf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0806e982-fe3c-496d-95e2-c75d2b23dacf-plugins-config-volume\") pod \"0806e982-fe3c-496d-95e2-c75d2b23dacf\" (UID: \"0806e982-fe3c-496d-95e2-c75d2b23dacf\") "
Oct 30 19:47:22.237411 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:22.237387    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0806e982-fe3c-496d-95e2-c75d2b23dacf-kube-api-access-tfn5d\" (UniqueName: \"kubernetes.io/projected/0806e982-fe3c-496d-95e2-c75d2b23dacf-kube-api-access-tfn5d\") pod \"0806e982-fe3c-496d-95e2-c75d2b23dacf\" (UID: \"0806e982-fe3c-496d-95e2-c75d2b23dacf\") "
Oct 30 19:47:22.237578 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:22.237555    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0806e982-fe3c-496d-95e2-c75d2b23dacf-plugins-config-volume" pod "0806e982-fe3c-496d-95e2-c75d2b23dacf" (UID: "0806e982-fe3c-496d-95e2-c75d2b23dacf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:22.237983 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:22.237957    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0806e982-fe3c-496d-95e2-c75d2b23dacf-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:22.239672 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:22.239647    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0806e982-fe3c-496d-95e2-c75d2b23dacf-kube-api-access-tfn5d" pod "0806e982-fe3c-496d-95e2-c75d2b23dacf" (UID: "0806e982-fe3c-496d-95e2-c75d2b23dacf"). InnerVolumeSpecName "kube-api-access-tfn5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:22.339468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:22.339391    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfn5d\" (UniqueName: \"kubernetes.io/projected/0806e982-fe3c-496d-95e2-c75d2b23dacf-kube-api-access-tfn5d\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:22.344418 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:22.344390    4524 generic.go:356] "Generic (PLEG): container finished" podID="0806e982-fe3c-496d-95e2-c75d2b23dacf" containerID="aa01e1114c8827bc3a74044e3300d71e54db7d06bdf336f5daa00afa3db99880" exitCode=0
Oct 30 19:47:22.344563 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:22.344443    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88"
Oct 30 19:47:22.344563 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:22.344469    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88" event={"ID":"0806e982-fe3c-496d-95e2-c75d2b23dacf","Type":"ContainerDied","Data":"aa01e1114c8827bc3a74044e3300d71e54db7d06bdf336f5daa00afa3db99880"}
Oct 30 19:47:22.344563 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:22.344556    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88" event={"ID":"0806e982-fe3c-496d-95e2-c75d2b23dacf","Type":"ContainerDied","Data":"384549fc4b4caf041bb8960f564edd12f9a5fd18eeebe7b172b41227dc2d60fe"}
Oct 30 19:47:22.344717 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:22.344580    4524 scope.go:122] "RemoveContainer" containerID="aa01e1114c8827bc3a74044e3300d71e54db7d06bdf336f5daa00afa3db99880"
Oct 30 19:47:22.356067 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:22.356050    4524 scope.go:122] "RemoveContainer" containerID="aa01e1114c8827bc3a74044e3300d71e54db7d06bdf336f5daa00afa3db99880"
Oct 30 19:47:22.356363 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:47:22.356345    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa01e1114c8827bc3a74044e3300d71e54db7d06bdf336f5daa00afa3db99880\": container with ID starting with aa01e1114c8827bc3a74044e3300d71e54db7d06bdf336f5daa00afa3db99880 not found: ID does not exist" containerID="aa01e1114c8827bc3a74044e3300d71e54db7d06bdf336f5daa00afa3db99880"
Oct 30 19:47:22.356436 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:22.356375    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa01e1114c8827bc3a74044e3300d71e54db7d06bdf336f5daa00afa3db99880"} err="failed to get container status \"aa01e1114c8827bc3a74044e3300d71e54db7d06bdf336f5daa00afa3db99880\": rpc error: code = NotFound desc = could not find container \"aa01e1114c8827bc3a74044e3300d71e54db7d06bdf336f5daa00afa3db99880\": container with ID starting with aa01e1114c8827bc3a74044e3300d71e54db7d06bdf336f5daa00afa3db99880 not found: ID does not exist"
Oct 30 19:47:22.391876 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:22.391849    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88"]
Oct 30 19:47:22.405679 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:22.405659    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-ad23c7-78895666b6-mcp88"]
Oct 30 19:47:23.443836 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:23.443760    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64" podUID="5d596f44-3f24-4f49-8baf-6dfe09ad625a" containerName="payload-pre-processing" containerID="cri-o://5dd78c97dcc0d3fa421d008ba41d3d9d00422138d235ed9a6188c5a9669eda52" gracePeriod=25
Oct 30 19:47:23.486502 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:23.486460    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64" podUID="5d596f44-3f24-4f49-8baf-6dfe09ad625a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:47:24.392072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:24.392040    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0806e982-fe3c-496d-95e2-c75d2b23dacf" path="/var/lib/kubelet/pods/0806e982-fe3c-496d-95e2-c75d2b23dacf/volumes"
Oct 30 19:47:24.610304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:24.610265    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"]
Oct 30 19:47:24.610794 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:24.610480    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422" podUID="54369e69-c268-48e4-80cd-84021afa2d10" containerName="istio-proxy" containerID="cri-o://946d7ea41696a16e1430476a4db868f5344e2d86d40bbc92dc5691a0736fab9b" gracePeriod=30
Oct 30 19:47:29.126722 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.126679    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"]
Oct 30 19:47:29.127257 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.126945    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql" podUID="881f121f-ca0c-4a99-a9d2-d0cca6b54a48" containerName="istio-proxy" containerID="cri-o://7758f773061f729e5d4ad5f10c00db3a296e448cb773107ceea5b7f2674ab3f7" gracePeriod=30
Oct 30 19:47:29.896857 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.896831    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:47:29.962744 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.962711    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54369e69-c268-48e4-80cd-84021afa2d10-istio-token\" (UniqueName: \"kubernetes.io/projected/54369e69-c268-48e4-80cd-84021afa2d10-istio-token\") pod \"54369e69-c268-48e4-80cd-84021afa2d10\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") "
Oct 30 19:47:29.962929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.962763    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-istio-data\") pod \"54369e69-c268-48e4-80cd-84021afa2d10\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") "
Oct 30 19:47:29.962929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.962801    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54369e69-c268-48e4-80cd-84021afa2d10-kube-api-access-pl6nj\" (UniqueName: \"kubernetes.io/projected/54369e69-c268-48e4-80cd-84021afa2d10-kube-api-access-pl6nj\") pod \"54369e69-c268-48e4-80cd-84021afa2d10\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") "
Oct 30 19:47:29.962929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.962834    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-credential-socket\") pod \"54369e69-c268-48e4-80cd-84021afa2d10\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") "
Oct 30 19:47:29.962929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.962881    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-istio-envoy\") pod \"54369e69-c268-48e4-80cd-84021afa2d10\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") "
Oct 30 19:47:29.963148 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.962931    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/54369e69-c268-48e4-80cd-84021afa2d10-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54369e69-c268-48e4-80cd-84021afa2d10-istio-podinfo\") pod \"54369e69-c268-48e4-80cd-84021afa2d10\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") "
Oct 30 19:47:29.963148 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.962966    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-workload-socket\") pod \"54369e69-c268-48e4-80cd-84021afa2d10\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") "
Oct 30 19:47:29.963148 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.963103    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/54369e69-c268-48e4-80cd-84021afa2d10-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54369e69-c268-48e4-80cd-84021afa2d10-istiod-ca-cert\") pod \"54369e69-c268-48e4-80cd-84021afa2d10\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") "
Oct 30 19:47:29.963346 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.963142    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-credential-socket" pod "54369e69-c268-48e4-80cd-84021afa2d10" (UID: "54369e69-c268-48e4-80cd-84021afa2d10"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:29.963346 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.963164    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-istio-data" pod "54369e69-c268-48e4-80cd-84021afa2d10" (UID: "54369e69-c268-48e4-80cd-84021afa2d10"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:29.963346 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.963324    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-workload-certs\") pod \"54369e69-c268-48e4-80cd-84021afa2d10\" (UID: \"54369e69-c268-48e4-80cd-84021afa2d10\") "
Oct 30 19:47:29.963504 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.963365    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-workload-socket" pod "54369e69-c268-48e4-80cd-84021afa2d10" (UID: "54369e69-c268-48e4-80cd-84021afa2d10"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:29.963578 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.963548    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54369e69-c268-48e4-80cd-84021afa2d10-istiod-ca-cert" pod "54369e69-c268-48e4-80cd-84021afa2d10" (UID: "54369e69-c268-48e4-80cd-84021afa2d10"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:29.963831 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.963795    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-workload-certs" pod "54369e69-c268-48e4-80cd-84021afa2d10" (UID: "54369e69-c268-48e4-80cd-84021afa2d10"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:29.964053 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.964027    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54369e69-c268-48e4-80cd-84021afa2d10-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:29.964153 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.964060    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:29.964153 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.964077    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:29.964153 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.964093    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:29.964153 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.964108    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:29.965721 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.965596    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54369e69-c268-48e4-80cd-84021afa2d10-istio-token" pod "54369e69-c268-48e4-80cd-84021afa2d10" (UID: "54369e69-c268-48e4-80cd-84021afa2d10"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:29.965721 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.965693    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/54369e69-c268-48e4-80cd-84021afa2d10-istio-podinfo" pod "54369e69-c268-48e4-80cd-84021afa2d10" (UID: "54369e69-c268-48e4-80cd-84021afa2d10"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:47:29.965721 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.965710    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-istio-envoy" pod "54369e69-c268-48e4-80cd-84021afa2d10" (UID: "54369e69-c268-48e4-80cd-84021afa2d10"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:29.966507 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:29.966477    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54369e69-c268-48e4-80cd-84021afa2d10-kube-api-access-pl6nj" pod "54369e69-c268-48e4-80cd-84021afa2d10" (UID: "54369e69-c268-48e4-80cd-84021afa2d10"). InnerVolumeSpecName "kube-api-access-pl6nj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:30.065975 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:30.065936    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54369e69-c268-48e4-80cd-84021afa2d10-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:30.065975 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:30.065968    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pl6nj\" (UniqueName: \"kubernetes.io/projected/54369e69-c268-48e4-80cd-84021afa2d10-kube-api-access-pl6nj\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:30.065975 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:30.065980    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54369e69-c268-48e4-80cd-84021afa2d10-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:30.066403 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:30.065989    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54369e69-c268-48e4-80cd-84021afa2d10-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:30.459905 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:30.459871    4524 generic.go:356] "Generic (PLEG): container finished" podID="54369e69-c268-48e4-80cd-84021afa2d10" containerID="946d7ea41696a16e1430476a4db868f5344e2d86d40bbc92dc5691a0736fab9b" exitCode=0
Oct 30 19:47:30.460340 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:30.459917    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"
Oct 30 19:47:30.460340 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:30.459951    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422" event={"ID":"54369e69-c268-48e4-80cd-84021afa2d10","Type":"ContainerDied","Data":"946d7ea41696a16e1430476a4db868f5344e2d86d40bbc92dc5691a0736fab9b"}
Oct 30 19:47:30.460340 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:30.460002    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422" event={"ID":"54369e69-c268-48e4-80cd-84021afa2d10","Type":"ContainerDied","Data":"8dee0d42d99f6c793de8528031e397426c8a76d35d94a735eb7af0947bf9a0cb"}
Oct 30 19:47:30.460340 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:30.460026    4524 scope.go:122] "RemoveContainer" containerID="946d7ea41696a16e1430476a4db868f5344e2d86d40bbc92dc5691a0736fab9b"
Oct 30 19:47:30.472671 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:30.472653    4524 scope.go:122] "RemoveContainer" containerID="946d7ea41696a16e1430476a4db868f5344e2d86d40bbc92dc5691a0736fab9b"
Oct 30 19:47:30.472945 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:47:30.472926    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"946d7ea41696a16e1430476a4db868f5344e2d86d40bbc92dc5691a0736fab9b\": container with ID starting with 946d7ea41696a16e1430476a4db868f5344e2d86d40bbc92dc5691a0736fab9b not found: ID does not exist" containerID="946d7ea41696a16e1430476a4db868f5344e2d86d40bbc92dc5691a0736fab9b"
Oct 30 19:47:30.473007 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:30.472953    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"946d7ea41696a16e1430476a4db868f5344e2d86d40bbc92dc5691a0736fab9b"} err="failed to get container status \"946d7ea41696a16e1430476a4db868f5344e2d86d40bbc92dc5691a0736fab9b\": rpc error: code = NotFound desc = could not find container \"946d7ea41696a16e1430476a4db868f5344e2d86d40bbc92dc5691a0736fab9b\": container with ID starting with 946d7ea41696a16e1430476a4db868f5344e2d86d40bbc92dc5691a0736fab9b not found: ID does not exist"
Oct 30 19:47:30.485845 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:30.485819    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"]
Oct 30 19:47:30.500065 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:30.500042    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ad23c7-openshift-default-7899f5cb8f-77422"]
Oct 30 19:47:30.827355 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:30.827272    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"]
Oct 30 19:47:30.827555 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:30.827523    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk" podUID="067cde38-0f2e-432e-b278-415e4ce5ff86" containerName="maas-api" containerID="cri-o://b8bad8cd4690ec5e3a8ce00d377cac49d15254e5e285268d0ac594d363a98c32" gracePeriod=30
Oct 30 19:47:31.357031 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.356997    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"
Oct 30 19:47:31.425069 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.425026    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/067cde38-0f2e-432e-b278-415e4ce5ff86-metrics-certs\" (UniqueName: \"kubernetes.io/secret/067cde38-0f2e-432e-b278-415e4ce5ff86-metrics-certs\") pod \"067cde38-0f2e-432e-b278-415e4ce5ff86\" (UID: \"067cde38-0f2e-432e-b278-415e4ce5ff86\") "
Oct 30 19:47:31.425278 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.425105    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/067cde38-0f2e-432e-b278-415e4ce5ff86-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/067cde38-0f2e-432e-b278-415e4ce5ff86-maas-api-tls\") pod \"067cde38-0f2e-432e-b278-415e4ce5ff86\" (UID: \"067cde38-0f2e-432e-b278-415e4ce5ff86\") "
Oct 30 19:47:31.425358 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.425316    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/067cde38-0f2e-432e-b278-415e4ce5ff86-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/067cde38-0f2e-432e-b278-415e4ce5ff86-openshift-service-ca\") pod \"067cde38-0f2e-432e-b278-415e4ce5ff86\" (UID: \"067cde38-0f2e-432e-b278-415e4ce5ff86\") "
Oct 30 19:47:31.425482 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.425408    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/067cde38-0f2e-432e-b278-415e4ce5ff86-kube-api-access-p8thd\" (UniqueName: \"kubernetes.io/projected/067cde38-0f2e-432e-b278-415e4ce5ff86-kube-api-access-p8thd\") pod \"067cde38-0f2e-432e-b278-415e4ce5ff86\" (UID: \"067cde38-0f2e-432e-b278-415e4ce5ff86\") "
Oct 30 19:47:31.426129 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.426097    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/067cde38-0f2e-432e-b278-415e4ce5ff86-openshift-service-ca" pod "067cde38-0f2e-432e-b278-415e4ce5ff86" (UID: "067cde38-0f2e-432e-b278-415e4ce5ff86"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:31.428422 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.428326    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/067cde38-0f2e-432e-b278-415e4ce5ff86-maas-api-tls" pod "067cde38-0f2e-432e-b278-415e4ce5ff86" (UID: "067cde38-0f2e-432e-b278-415e4ce5ff86"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:31.428422 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.428377    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/067cde38-0f2e-432e-b278-415e4ce5ff86-kube-api-access-p8thd" pod "067cde38-0f2e-432e-b278-415e4ce5ff86" (UID: "067cde38-0f2e-432e-b278-415e4ce5ff86"). InnerVolumeSpecName "kube-api-access-p8thd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:31.428422 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.428385    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/067cde38-0f2e-432e-b278-415e4ce5ff86-metrics-certs" pod "067cde38-0f2e-432e-b278-415e4ce5ff86" (UID: "067cde38-0f2e-432e-b278-415e4ce5ff86"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:31.468851 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.468810    4524 generic.go:356] "Generic (PLEG): container finished" podID="067cde38-0f2e-432e-b278-415e4ce5ff86" containerID="b8bad8cd4690ec5e3a8ce00d377cac49d15254e5e285268d0ac594d363a98c32" exitCode=0
Oct 30 19:47:31.469320 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.469010    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk" event={"ID":"067cde38-0f2e-432e-b278-415e4ce5ff86","Type":"ContainerDied","Data":"b8bad8cd4690ec5e3a8ce00d377cac49d15254e5e285268d0ac594d363a98c32"}
Oct 30 19:47:31.469320 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.469061    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk" event={"ID":"067cde38-0f2e-432e-b278-415e4ce5ff86","Type":"ContainerDied","Data":"3ec2e2e72225167faa011d9cdf5408e27237247fa7696bdfa7ace63ea0b82390"}
Oct 30 19:47:31.469320 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.469090    4524 scope.go:122] "RemoveContainer" containerID="b8bad8cd4690ec5e3a8ce00d377cac49d15254e5e285268d0ac594d363a98c32"
Oct 30 19:47:31.469320 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.469126    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"
Oct 30 19:47:31.481610 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.481592    4524 scope.go:122] "RemoveContainer" containerID="b8bad8cd4690ec5e3a8ce00d377cac49d15254e5e285268d0ac594d363a98c32"
Oct 30 19:47:31.481920 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:47:31.481895    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8bad8cd4690ec5e3a8ce00d377cac49d15254e5e285268d0ac594d363a98c32\": container with ID starting with b8bad8cd4690ec5e3a8ce00d377cac49d15254e5e285268d0ac594d363a98c32 not found: ID does not exist" containerID="b8bad8cd4690ec5e3a8ce00d377cac49d15254e5e285268d0ac594d363a98c32"
Oct 30 19:47:31.482045 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.481932    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8bad8cd4690ec5e3a8ce00d377cac49d15254e5e285268d0ac594d363a98c32"} err="failed to get container status \"b8bad8cd4690ec5e3a8ce00d377cac49d15254e5e285268d0ac594d363a98c32\": rpc error: code = NotFound desc = could not find container \"b8bad8cd4690ec5e3a8ce00d377cac49d15254e5e285268d0ac594d363a98c32\": container with ID starting with b8bad8cd4690ec5e3a8ce00d377cac49d15254e5e285268d0ac594d363a98c32 not found: ID does not exist"
Oct 30 19:47:31.516847 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.516716    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"]
Oct 30 19:47:31.528873 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.528844    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/067cde38-0f2e-432e-b278-415e4ce5ff86-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:31.528873 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.528875    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/067cde38-0f2e-432e-b278-415e4ce5ff86-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:31.529103 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.528913    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/067cde38-0f2e-432e-b278-415e4ce5ff86-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:31.529103 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.528930    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8thd\" (UniqueName: \"kubernetes.io/projected/067cde38-0f2e-432e-b278-415e4ce5ff86-kube-api-access-p8thd\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:31.533701 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:31.533673    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w4-8e6762-6fb99dfcf7-sxhtk"]
Oct 30 19:47:32.392869 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:32.392835    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="067cde38-0f2e-432e-b278-415e4ce5ff86" path="/var/lib/kubelet/pods/067cde38-0f2e-432e-b278-415e4ce5ff86/volumes"
Oct 30 19:47:32.393518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:32.393493    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="54369e69-c268-48e4-80cd-84021afa2d10" path="/var/lib/kubelet/pods/54369e69-c268-48e4-80cd-84021afa2d10/volumes"
Oct 30 19:47:33.486267 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:33.486220    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64" podUID="5d596f44-3f24-4f49-8baf-6dfe09ad625a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:47:34.452808 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.452778    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:47:34.502955 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.502922    4524 generic.go:356] "Generic (PLEG): container finished" podID="881f121f-ca0c-4a99-a9d2-d0cca6b54a48" containerID="7758f773061f729e5d4ad5f10c00db3a296e448cb773107ceea5b7f2674ab3f7" exitCode=0
Oct 30 19:47:34.503430 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.503139    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql" event={"ID":"881f121f-ca0c-4a99-a9d2-d0cca6b54a48","Type":"ContainerDied","Data":"7758f773061f729e5d4ad5f10c00db3a296e448cb773107ceea5b7f2674ab3f7"}
Oct 30 19:47:34.503430 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.503225    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql" event={"ID":"881f121f-ca0c-4a99-a9d2-d0cca6b54a48","Type":"ContainerDied","Data":"f157d8fe0190f7f3d7d73c4f5c6fad062f30a9b12defd6d2448016927953a8a4"}
Oct 30 19:47:34.503430 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.503260    4524 scope.go:122] "RemoveContainer" containerID="7758f773061f729e5d4ad5f10c00db3a296e448cb773107ceea5b7f2674ab3f7"
Oct 30 19:47:34.503430 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.503418    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"
Oct 30 19:47:34.523160 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.523129    4524 scope.go:122] "RemoveContainer" containerID="905aaceacb19f21070b83bc9cfd84e72fc956abb656dc5f788f154c479933cd5"
Oct 30 19:47:34.538562 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.538538    4524 scope.go:122] "RemoveContainer" containerID="7758f773061f729e5d4ad5f10c00db3a296e448cb773107ceea5b7f2674ab3f7"
Oct 30 19:47:34.538894 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:47:34.538868    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7758f773061f729e5d4ad5f10c00db3a296e448cb773107ceea5b7f2674ab3f7\": container with ID starting with 7758f773061f729e5d4ad5f10c00db3a296e448cb773107ceea5b7f2674ab3f7 not found: ID does not exist" containerID="7758f773061f729e5d4ad5f10c00db3a296e448cb773107ceea5b7f2674ab3f7"
Oct 30 19:47:34.539010 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.538905    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7758f773061f729e5d4ad5f10c00db3a296e448cb773107ceea5b7f2674ab3f7"} err="failed to get container status \"7758f773061f729e5d4ad5f10c00db3a296e448cb773107ceea5b7f2674ab3f7\": rpc error: code = NotFound desc = could not find container \"7758f773061f729e5d4ad5f10c00db3a296e448cb773107ceea5b7f2674ab3f7\": container with ID starting with 7758f773061f729e5d4ad5f10c00db3a296e448cb773107ceea5b7f2674ab3f7 not found: ID does not exist"
Oct 30 19:47:34.539010 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.538930    4524 scope.go:122] "RemoveContainer" containerID="905aaceacb19f21070b83bc9cfd84e72fc956abb656dc5f788f154c479933cd5"
Oct 30 19:47:34.539234 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:47:34.539210    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"905aaceacb19f21070b83bc9cfd84e72fc956abb656dc5f788f154c479933cd5\": container with ID starting with 905aaceacb19f21070b83bc9cfd84e72fc956abb656dc5f788f154c479933cd5 not found: ID does not exist" containerID="905aaceacb19f21070b83bc9cfd84e72fc956abb656dc5f788f154c479933cd5"
Oct 30 19:47:34.539336 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.539242    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"905aaceacb19f21070b83bc9cfd84e72fc956abb656dc5f788f154c479933cd5"} err="failed to get container status \"905aaceacb19f21070b83bc9cfd84e72fc956abb656dc5f788f154c479933cd5\": rpc error: code = NotFound desc = could not find container \"905aaceacb19f21070b83bc9cfd84e72fc956abb656dc5f788f154c479933cd5\": container with ID starting with 905aaceacb19f21070b83bc9cfd84e72fc956abb656dc5f788f154c479933cd5 not found: ID does not exist"
Oct 30 19:47:34.570324 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.570303    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-data\") pod \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") "
Oct 30 19:47:34.570427 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.570336    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-workload-certs\") pod \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") "
Oct 30 19:47:34.570427 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.570394    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-token\" (UniqueName: \"kubernetes.io/projected/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-token\") pod \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") "
Oct 30 19:47:34.570427 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.570417    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istiod-ca-cert\") pod \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") "
Oct 30 19:47:34.570583 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.570438    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-workload-socket\") pod \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") "
Oct 30 19:47:34.570641 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.570594    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-envoy\") pod \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") "
Oct 30 19:47:34.570641 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.570603    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-data" pod "881f121f-ca0c-4a99-a9d2-d0cca6b54a48" (UID: "881f121f-ca0c-4a99-a9d2-d0cca6b54a48"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:34.571389 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.570651    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-credential-socket\") pod \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") "
Oct 30 19:47:34.571389 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.570700    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-workload-certs" pod "881f121f-ca0c-4a99-a9d2-d0cca6b54a48" (UID: "881f121f-ca0c-4a99-a9d2-d0cca6b54a48"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:34.571389 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.570742    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-podinfo\") pod \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") "
Oct 30 19:47:34.571389 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.570779    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-kube-api-access-g4jxt\" (UniqueName: \"kubernetes.io/projected/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-kube-api-access-g4jxt\") pod \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\" (UID: \"881f121f-ca0c-4a99-a9d2-d0cca6b54a48\") "
Oct 30 19:47:34.571389 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.570824    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-workload-socket" pod "881f121f-ca0c-4a99-a9d2-d0cca6b54a48" (UID: "881f121f-ca0c-4a99-a9d2-d0cca6b54a48"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:34.571389 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.570911    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-credential-socket" pod "881f121f-ca0c-4a99-a9d2-d0cca6b54a48" (UID: "881f121f-ca0c-4a99-a9d2-d0cca6b54a48"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:34.571949 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.571392    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:34.571949 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.571414    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:34.571949 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.571429    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:34.571949 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.571443    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:34.571949 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.571877    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istiod-ca-cert" pod "881f121f-ca0c-4a99-a9d2-d0cca6b54a48" (UID: "881f121f-ca0c-4a99-a9d2-d0cca6b54a48"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:34.573551 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.573499    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-token" pod "881f121f-ca0c-4a99-a9d2-d0cca6b54a48" (UID: "881f121f-ca0c-4a99-a9d2-d0cca6b54a48"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:34.573681 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.573644    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-kube-api-access-g4jxt" pod "881f121f-ca0c-4a99-a9d2-d0cca6b54a48" (UID: "881f121f-ca0c-4a99-a9d2-d0cca6b54a48"). InnerVolumeSpecName "kube-api-access-g4jxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:34.573681 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.573657    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-envoy" pod "881f121f-ca0c-4a99-a9d2-d0cca6b54a48" (UID: "881f121f-ca0c-4a99-a9d2-d0cca6b54a48"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:34.574004 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.573891    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-podinfo" pod "881f121f-ca0c-4a99-a9d2-d0cca6b54a48" (UID: "881f121f-ca0c-4a99-a9d2-d0cca6b54a48"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:47:34.672367 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.672300    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:34.672367 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.672329    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:34.672367 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.672338    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:34.672367 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.672349    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:34.672367 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.672359    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4jxt\" (UniqueName: \"kubernetes.io/projected/881f121f-ca0c-4a99-a9d2-d0cca6b54a48-kube-api-access-g4jxt\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:34.853678 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.853649    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"]
Oct 30 19:47:34.859969 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:34.859943    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-7aa217-openshift-default-777b88d9b9-z5kql"]
Oct 30 19:47:36.407633 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:36.407598    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="881f121f-ca0c-4a99-a9d2-d0cca6b54a48" path="/var/lib/kubelet/pods/881f121f-ca0c-4a99-a9d2-d0cca6b54a48/volumes"
Oct 30 19:47:40.924318 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:40.924247    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"]
Oct 30 19:47:40.924875 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:40.924491    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq" podUID="1b05d10a-5c87-42d8-8880-2511d27bf716" containerName="istio-proxy" containerID="cri-o://ae5ced2f8ce1f3db2704d0becb437d38dd6f547113f9094b3b527bb4c75a10c5" gracePeriod=30
Oct 30 19:47:43.485178 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:47:43.485141    4524 logging.go:55] [core] [Channel #7621 SubChannel #7622]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.152:50052", ServerName: "10.133.0.152:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.152:50052: connect: connection refused"
Oct 30 19:47:43.604547 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:43.604506    4524 generic.go:356] "Generic (PLEG): container finished" podID="5d596f44-3f24-4f49-8baf-6dfe09ad625a" containerID="5dd78c97dcc0d3fa421d008ba41d3d9d00422138d235ed9a6188c5a9669eda52" exitCode=0
Oct 30 19:47:43.604752 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:43.604723    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64" event={"ID":"5d596f44-3f24-4f49-8baf-6dfe09ad625a","Type":"ContainerDied","Data":"5dd78c97dcc0d3fa421d008ba41d3d9d00422138d235ed9a6188c5a9669eda52"}
Oct 30 19:47:43.795236 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:43.795209    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64"
Oct 30 19:47:43.849343 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:43.849313    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5d596f44-3f24-4f49-8baf-6dfe09ad625a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d596f44-3f24-4f49-8baf-6dfe09ad625a-plugins-config-volume\") pod \"5d596f44-3f24-4f49-8baf-6dfe09ad625a\" (UID: \"5d596f44-3f24-4f49-8baf-6dfe09ad625a\") "
Oct 30 19:47:43.849517 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:43.849370    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d596f44-3f24-4f49-8baf-6dfe09ad625a-kube-api-access-bh2sl\" (UniqueName: \"kubernetes.io/projected/5d596f44-3f24-4f49-8baf-6dfe09ad625a-kube-api-access-bh2sl\") pod \"5d596f44-3f24-4f49-8baf-6dfe09ad625a\" (UID: \"5d596f44-3f24-4f49-8baf-6dfe09ad625a\") "
Oct 30 19:47:43.849792 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:43.849761    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d596f44-3f24-4f49-8baf-6dfe09ad625a-plugins-config-volume" pod "5d596f44-3f24-4f49-8baf-6dfe09ad625a" (UID: "5d596f44-3f24-4f49-8baf-6dfe09ad625a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:43.850036 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:43.850014    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d596f44-3f24-4f49-8baf-6dfe09ad625a-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:43.851678 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:43.851648    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d596f44-3f24-4f49-8baf-6dfe09ad625a-kube-api-access-bh2sl" pod "5d596f44-3f24-4f49-8baf-6dfe09ad625a" (UID: "5d596f44-3f24-4f49-8baf-6dfe09ad625a"). InnerVolumeSpecName "kube-api-access-bh2sl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:43.951016 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:43.950977    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bh2sl\" (UniqueName: \"kubernetes.io/projected/5d596f44-3f24-4f49-8baf-6dfe09ad625a-kube-api-access-bh2sl\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:44.505897 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:44.504660    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64" podUID="5d596f44-3f24-4f49-8baf-6dfe09ad625a" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.133.0.152:50052\" within 1s: context deadline exceeded"
Oct 30 19:47:44.505897 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:44.504774    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64"
Oct 30 19:47:44.505897 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:47:44.504904    4524 logging.go:55] [core] [Channel #7621 SubChannel #7622]grpc: addrConn.createTransport failed to connect to {Addr: "10.133.0.152:50052", ServerName: "10.133.0.152:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.133.0.152:50052: operation was canceled"
Oct 30 19:47:44.613382 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:44.613353    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64"
Oct 30 19:47:44.613590 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:44.613386    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64" event={"ID":"5d596f44-3f24-4f49-8baf-6dfe09ad625a","Type":"ContainerDied","Data":"a356a966cfc08270649ff718d2fe7a243da30d2c89f833de015cabd052aa0596"}
Oct 30 19:47:44.613590 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:44.613458    4524 scope.go:122] "RemoveContainer" containerID="5dd78c97dcc0d3fa421d008ba41d3d9d00422138d235ed9a6188c5a9669eda52"
Oct 30 19:47:44.640426 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:44.640390    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64"]
Oct 30 19:47:44.644270 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:44.644249    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-8e6762-5c986d5695-jmg64"]
Oct 30 19:47:45.628274 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:47:45.628239    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:47:46.201727 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.201644    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"]
Oct 30 19:47:46.202231 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.201925    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4" podUID="1e3ef760-19e1-45bc-b806-57d79f5082db" containerName="istio-proxy" containerID="cri-o://3dcd0747cf48ef6b9a8c3a58428d58adc28f9995be16d70bc4709ade02733049" gracePeriod=30
Oct 30 19:47:46.261907 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.261875    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:47:46.281576 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.281544    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-token\" (UniqueName: \"kubernetes.io/projected/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-token\") pod \"1b05d10a-5c87-42d8-8880-2511d27bf716\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") "
Oct 30 19:47:46.281722 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.281600    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-data\") pod \"1b05d10a-5c87-42d8-8880-2511d27bf716\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") "
Oct 30 19:47:46.281914 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.281895    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-podinfo\") pod \"1b05d10a-5c87-42d8-8880-2511d27bf716\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") "
Oct 30 19:47:46.282012 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.282002    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b05d10a-5c87-42d8-8880-2511d27bf716-kube-api-access-st2kj\" (UniqueName: \"kubernetes.io/projected/1b05d10a-5c87-42d8-8880-2511d27bf716-kube-api-access-st2kj\") pod \"1b05d10a-5c87-42d8-8880-2511d27bf716\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") "
Oct 30 19:47:46.282083 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.282028    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-envoy\") pod \"1b05d10a-5c87-42d8-8880-2511d27bf716\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") "
Oct 30 19:47:46.282083 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.282047    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-credential-socket\") pod \"1b05d10a-5c87-42d8-8880-2511d27bf716\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") "
Oct 30 19:47:46.282225 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.282099    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b05d10a-5c87-42d8-8880-2511d27bf716-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b05d10a-5c87-42d8-8880-2511d27bf716-istiod-ca-cert\") pod \"1b05d10a-5c87-42d8-8880-2511d27bf716\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") "
Oct 30 19:47:46.282225 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.282128    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-workload-certs\") pod \"1b05d10a-5c87-42d8-8880-2511d27bf716\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") "
Oct 30 19:47:46.282225 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.282158    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-workload-socket\") pod \"1b05d10a-5c87-42d8-8880-2511d27bf716\" (UID: \"1b05d10a-5c87-42d8-8880-2511d27bf716\") "
Oct 30 19:47:46.282393 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.282269    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-data" pod "1b05d10a-5c87-42d8-8880-2511d27bf716" (UID: "1b05d10a-5c87-42d8-8880-2511d27bf716"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:46.282502 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.282482    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-credential-socket" pod "1b05d10a-5c87-42d8-8880-2511d27bf716" (UID: "1b05d10a-5c87-42d8-8880-2511d27bf716"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:46.282734 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.282708    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:46.282809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.282741    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:46.283058 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.283035    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b05d10a-5c87-42d8-8880-2511d27bf716-istiod-ca-cert" pod "1b05d10a-5c87-42d8-8880-2511d27bf716" (UID: "1b05d10a-5c87-42d8-8880-2511d27bf716"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:46.283456 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.283307    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-workload-certs" pod "1b05d10a-5c87-42d8-8880-2511d27bf716" (UID: "1b05d10a-5c87-42d8-8880-2511d27bf716"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:46.283611 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.283536    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-workload-socket" pod "1b05d10a-5c87-42d8-8880-2511d27bf716" (UID: "1b05d10a-5c87-42d8-8880-2511d27bf716"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:46.285625 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.285593    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b05d10a-5c87-42d8-8880-2511d27bf716-kube-api-access-st2kj" pod "1b05d10a-5c87-42d8-8880-2511d27bf716" (UID: "1b05d10a-5c87-42d8-8880-2511d27bf716"). InnerVolumeSpecName "kube-api-access-st2kj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:46.285941 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.285904    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-token" pod "1b05d10a-5c87-42d8-8880-2511d27bf716" (UID: "1b05d10a-5c87-42d8-8880-2511d27bf716"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:46.286424 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.286402    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-envoy" pod "1b05d10a-5c87-42d8-8880-2511d27bf716" (UID: "1b05d10a-5c87-42d8-8880-2511d27bf716"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:46.374394 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.325314    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-podinfo" pod "1b05d10a-5c87-42d8-8880-2511d27bf716" (UID: "1b05d10a-5c87-42d8-8880-2511d27bf716"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:47:46.423468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.422363    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:46.423468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.422402    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-st2kj\" (UniqueName: \"kubernetes.io/projected/1b05d10a-5c87-42d8-8880-2511d27bf716-kube-api-access-st2kj\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:46.423468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.422422    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:46.423468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.422439    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b05d10a-5c87-42d8-8880-2511d27bf716-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:46.423468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.422453    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:46.423468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.422467    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b05d10a-5c87-42d8-8880-2511d27bf716-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:46.423468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.422480    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b05d10a-5c87-42d8-8880-2511d27bf716-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:46.445265 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.445228    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5d596f44-3f24-4f49-8baf-6dfe09ad625a" path="/var/lib/kubelet/pods/5d596f44-3f24-4f49-8baf-6dfe09ad625a/volumes"
Oct 30 19:47:46.641045 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.641005    4524 generic.go:356] "Generic (PLEG): container finished" podID="1b05d10a-5c87-42d8-8880-2511d27bf716" containerID="ae5ced2f8ce1f3db2704d0becb437d38dd6f547113f9094b3b527bb4c75a10c5" exitCode=0
Oct 30 19:47:46.641603 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.641056    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq" event={"ID":"1b05d10a-5c87-42d8-8880-2511d27bf716","Type":"ContainerDied","Data":"ae5ced2f8ce1f3db2704d0becb437d38dd6f547113f9094b3b527bb4c75a10c5"}
Oct 30 19:47:46.641603 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.641112    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"
Oct 30 19:47:46.641603 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.641138    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq" event={"ID":"1b05d10a-5c87-42d8-8880-2511d27bf716","Type":"ContainerDied","Data":"f868876d9b04b212155af1101e3ab6fd07a59665b55fe34a8caf57bc17f15305"}
Oct 30 19:47:46.641603 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.641195    4524 scope.go:122] "RemoveContainer" containerID="ae5ced2f8ce1f3db2704d0becb437d38dd6f547113f9094b3b527bb4c75a10c5"
Oct 30 19:47:46.660399 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.660349    4524 scope.go:122] "RemoveContainer" containerID="ae5ced2f8ce1f3db2704d0becb437d38dd6f547113f9094b3b527bb4c75a10c5"
Oct 30 19:47:46.660808 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:47:46.660780    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae5ced2f8ce1f3db2704d0becb437d38dd6f547113f9094b3b527bb4c75a10c5\": container with ID starting with ae5ced2f8ce1f3db2704d0becb437d38dd6f547113f9094b3b527bb4c75a10c5 not found: ID does not exist" containerID="ae5ced2f8ce1f3db2704d0becb437d38dd6f547113f9094b3b527bb4c75a10c5"
Oct 30 19:47:46.660939 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.660820    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae5ced2f8ce1f3db2704d0becb437d38dd6f547113f9094b3b527bb4c75a10c5"} err="failed to get container status \"ae5ced2f8ce1f3db2704d0becb437d38dd6f547113f9094b3b527bb4c75a10c5\": rpc error: code = NotFound desc = could not find container \"ae5ced2f8ce1f3db2704d0becb437d38dd6f547113f9094b3b527bb4c75a10c5\": container with ID starting with ae5ced2f8ce1f3db2704d0becb437d38dd6f547113f9094b3b527bb4c75a10c5 not found: ID does not exist"
Oct 30 19:47:46.664809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.664786    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"]
Oct 30 19:47:46.670047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:46.670026    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-8e6762-openshift-default-59b8fc5d79-9csxq"]
Oct 30 19:47:48.395543 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:48.395505    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b05d10a-5c87-42d8-8880-2511d27bf716" path="/var/lib/kubelet/pods/1b05d10a-5c87-42d8-8880-2511d27bf716/volumes"
Oct 30 19:47:50.242143 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.242113    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"]
Oct 30 19:47:50.242655 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.242379    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7" podUID="944c40c9-95ff-419a-8429-2e8c9a2eefba" containerName="maas-api" containerID="cri-o://5cfe27a6aabaaf14814af578b77b34e90f4c7a5a0884e179eba1ed6997455bf5" gracePeriod=30
Oct 30 19:47:50.679749 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.679716    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"
Oct 30 19:47:50.698911 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.698882    4524 generic.go:356] "Generic (PLEG): container finished" podID="944c40c9-95ff-419a-8429-2e8c9a2eefba" containerID="5cfe27a6aabaaf14814af578b77b34e90f4c7a5a0884e179eba1ed6997455bf5" exitCode=0
Oct 30 19:47:50.699088 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.699075    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7" event={"ID":"944c40c9-95ff-419a-8429-2e8c9a2eefba","Type":"ContainerDied","Data":"5cfe27a6aabaaf14814af578b77b34e90f4c7a5a0884e179eba1ed6997455bf5"}
Oct 30 19:47:50.699138 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.699116    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7" event={"ID":"944c40c9-95ff-419a-8429-2e8c9a2eefba","Type":"ContainerDied","Data":"6cc4b5efe829ddc10d085c24c0f9e0a4a854813ed965c65789025707e04ab169"}
Oct 30 19:47:50.699204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.699137    4524 scope.go:122] "RemoveContainer" containerID="5cfe27a6aabaaf14814af578b77b34e90f4c7a5a0884e179eba1ed6997455bf5"
Oct 30 19:47:50.699266 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.699258    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"
Oct 30 19:47:50.712533 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.712230    4524 scope.go:122] "RemoveContainer" containerID="5cfe27a6aabaaf14814af578b77b34e90f4c7a5a0884e179eba1ed6997455bf5"
Oct 30 19:47:50.712629 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:47:50.712587    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cfe27a6aabaaf14814af578b77b34e90f4c7a5a0884e179eba1ed6997455bf5\": container with ID starting with 5cfe27a6aabaaf14814af578b77b34e90f4c7a5a0884e179eba1ed6997455bf5 not found: ID does not exist" containerID="5cfe27a6aabaaf14814af578b77b34e90f4c7a5a0884e179eba1ed6997455bf5"
Oct 30 19:47:50.712674 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.712639    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cfe27a6aabaaf14814af578b77b34e90f4c7a5a0884e179eba1ed6997455bf5"} err="failed to get container status \"5cfe27a6aabaaf14814af578b77b34e90f4c7a5a0884e179eba1ed6997455bf5\": rpc error: code = NotFound desc = could not find container \"5cfe27a6aabaaf14814af578b77b34e90f4c7a5a0884e179eba1ed6997455bf5\": container with ID starting with 5cfe27a6aabaaf14814af578b77b34e90f4c7a5a0884e179eba1ed6997455bf5 not found: ID does not exist"
Oct 30 19:47:50.785554 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.785520    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/944c40c9-95ff-419a-8429-2e8c9a2eefba-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/944c40c9-95ff-419a-8429-2e8c9a2eefba-maas-api-tls\") pod \"944c40c9-95ff-419a-8429-2e8c9a2eefba\" (UID: \"944c40c9-95ff-419a-8429-2e8c9a2eefba\") "
Oct 30 19:47:50.785751 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.785627    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/944c40c9-95ff-419a-8429-2e8c9a2eefba-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/944c40c9-95ff-419a-8429-2e8c9a2eefba-openshift-service-ca\") pod \"944c40c9-95ff-419a-8429-2e8c9a2eefba\" (UID: \"944c40c9-95ff-419a-8429-2e8c9a2eefba\") "
Oct 30 19:47:50.785751 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.785707    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/944c40c9-95ff-419a-8429-2e8c9a2eefba-metrics-certs\" (UniqueName: \"kubernetes.io/secret/944c40c9-95ff-419a-8429-2e8c9a2eefba-metrics-certs\") pod \"944c40c9-95ff-419a-8429-2e8c9a2eefba\" (UID: \"944c40c9-95ff-419a-8429-2e8c9a2eefba\") "
Oct 30 19:47:50.785871 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.785809    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/944c40c9-95ff-419a-8429-2e8c9a2eefba-kube-api-access-bw6kq\" (UniqueName: \"kubernetes.io/projected/944c40c9-95ff-419a-8429-2e8c9a2eefba-kube-api-access-bw6kq\") pod \"944c40c9-95ff-419a-8429-2e8c9a2eefba\" (UID: \"944c40c9-95ff-419a-8429-2e8c9a2eefba\") "
Oct 30 19:47:50.786036 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.786006    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/944c40c9-95ff-419a-8429-2e8c9a2eefba-openshift-service-ca" pod "944c40c9-95ff-419a-8429-2e8c9a2eefba" (UID: "944c40c9-95ff-419a-8429-2e8c9a2eefba"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:50.786495 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.786470    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/944c40c9-95ff-419a-8429-2e8c9a2eefba-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:50.787915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.787890    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/944c40c9-95ff-419a-8429-2e8c9a2eefba-maas-api-tls" pod "944c40c9-95ff-419a-8429-2e8c9a2eefba" (UID: "944c40c9-95ff-419a-8429-2e8c9a2eefba"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:50.788032 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.787941    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/944c40c9-95ff-419a-8429-2e8c9a2eefba-metrics-certs" pod "944c40c9-95ff-419a-8429-2e8c9a2eefba" (UID: "944c40c9-95ff-419a-8429-2e8c9a2eefba"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:50.788423 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.788399    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/944c40c9-95ff-419a-8429-2e8c9a2eefba-kube-api-access-bw6kq" pod "944c40c9-95ff-419a-8429-2e8c9a2eefba" (UID: "944c40c9-95ff-419a-8429-2e8c9a2eefba"). InnerVolumeSpecName "kube-api-access-bw6kq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:50.909316 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.893790    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/944c40c9-95ff-419a-8429-2e8c9a2eefba-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:50.909316 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.893826    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bw6kq\" (UniqueName: \"kubernetes.io/projected/944c40c9-95ff-419a-8429-2e8c9a2eefba-kube-api-access-bw6kq\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:50.909316 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:50.893840    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/944c40c9-95ff-419a-8429-2e8c9a2eefba-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:51.023820 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.023788    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"]
Oct 30 19:47:51.039632 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.039600    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-aa49c5-676c489fbc-vxxf7"]
Oct 30 19:47:51.068076 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.068044    4524 patch_prober.go:29] interesting pod/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:47:51.068270 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.068104    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4" podUID="1e3ef760-19e1-45bc-b806-57d79f5082db" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:47:51.507039 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.507013    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:47:51.629929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.619694    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1e3ef760-19e1-45bc-b806-57d79f5082db-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e3ef760-19e1-45bc-b806-57d79f5082db-istiod-ca-cert\") pod \"1e3ef760-19e1-45bc-b806-57d79f5082db\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") "
Oct 30 19:47:51.629929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.619754    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-data\") pod \"1e3ef760-19e1-45bc-b806-57d79f5082db\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") "
Oct 30 19:47:51.629929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.619838    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-podinfo\") pod \"1e3ef760-19e1-45bc-b806-57d79f5082db\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") "
Oct 30 19:47:51.629929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.619882    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-workload-socket\") pod \"1e3ef760-19e1-45bc-b806-57d79f5082db\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") "
Oct 30 19:47:51.629929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.619924    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e3ef760-19e1-45bc-b806-57d79f5082db-kube-api-access-pfkk9\" (UniqueName: \"kubernetes.io/projected/1e3ef760-19e1-45bc-b806-57d79f5082db-kube-api-access-pfkk9\") pod \"1e3ef760-19e1-45bc-b806-57d79f5082db\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") "
Oct 30 19:47:51.629929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.620027    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-workload-certs\") pod \"1e3ef760-19e1-45bc-b806-57d79f5082db\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") "
Oct 30 19:47:51.629929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.620104    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-token\" (UniqueName: \"kubernetes.io/projected/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-token\") pod \"1e3ef760-19e1-45bc-b806-57d79f5082db\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") "
Oct 30 19:47:51.629929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.620199    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-credential-socket\") pod \"1e3ef760-19e1-45bc-b806-57d79f5082db\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") "
Oct 30 19:47:51.629929 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.620293    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-envoy\") pod \"1e3ef760-19e1-45bc-b806-57d79f5082db\" (UID: \"1e3ef760-19e1-45bc-b806-57d79f5082db\") "
Oct 30 19:47:51.632987 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.632355    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e3ef760-19e1-45bc-b806-57d79f5082db-istiod-ca-cert" pod "1e3ef760-19e1-45bc-b806-57d79f5082db" (UID: "1e3ef760-19e1-45bc-b806-57d79f5082db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:51.632987 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.632574    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-workload-socket" pod "1e3ef760-19e1-45bc-b806-57d79f5082db" (UID: "1e3ef760-19e1-45bc-b806-57d79f5082db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:51.632987 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.632639    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-workload-certs" pod "1e3ef760-19e1-45bc-b806-57d79f5082db" (UID: "1e3ef760-19e1-45bc-b806-57d79f5082db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:51.632987 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.632820    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-data" pod "1e3ef760-19e1-45bc-b806-57d79f5082db" (UID: "1e3ef760-19e1-45bc-b806-57d79f5082db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:51.632987 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.632884    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-credential-socket" pod "1e3ef760-19e1-45bc-b806-57d79f5082db" (UID: "1e3ef760-19e1-45bc-b806-57d79f5082db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:51.636127 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.635531    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-token" pod "1e3ef760-19e1-45bc-b806-57d79f5082db" (UID: "1e3ef760-19e1-45bc-b806-57d79f5082db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:51.641420 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.637016    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-podinfo" pod "1e3ef760-19e1-45bc-b806-57d79f5082db" (UID: "1e3ef760-19e1-45bc-b806-57d79f5082db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:47:51.641420 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.638654    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e3ef760-19e1-45bc-b806-57d79f5082db-kube-api-access-pfkk9" pod "1e3ef760-19e1-45bc-b806-57d79f5082db" (UID: "1e3ef760-19e1-45bc-b806-57d79f5082db"). InnerVolumeSpecName "kube-api-access-pfkk9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:51.645720 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.645689    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-envoy" pod "1e3ef760-19e1-45bc-b806-57d79f5082db" (UID: "1e3ef760-19e1-45bc-b806-57d79f5082db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:47:51.714191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.713967    4524 generic.go:356] "Generic (PLEG): container finished" podID="1e3ef760-19e1-45bc-b806-57d79f5082db" containerID="3dcd0747cf48ef6b9a8c3a58428d58adc28f9995be16d70bc4709ade02733049" exitCode=0
Oct 30 19:47:51.714191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.714050    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4" event={"ID":"1e3ef760-19e1-45bc-b806-57d79f5082db","Type":"ContainerDied","Data":"3dcd0747cf48ef6b9a8c3a58428d58adc28f9995be16d70bc4709ade02733049"}
Oct 30 19:47:51.714191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.714132    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4" event={"ID":"1e3ef760-19e1-45bc-b806-57d79f5082db","Type":"ContainerDied","Data":"eb5aee72ae4d2c4d2679baaf5608fb3fce90b142121d6c0addff83acebceaa54"}
Oct 30 19:47:51.714191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.714196    4524 scope.go:122] "RemoveContainer" containerID="3dcd0747cf48ef6b9a8c3a58428d58adc28f9995be16d70bc4709ade02733049"
Oct 30 19:47:51.714545 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.714439    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"
Oct 30 19:47:51.722683 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.722643    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:51.722683 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.722680    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:51.722856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.722703    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:51.722856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.722718    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:51.722856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.722732    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e3ef760-19e1-45bc-b806-57d79f5082db-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:51.722856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.722746    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:51.722856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.722764    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e3ef760-19e1-45bc-b806-57d79f5082db-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:51.722856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.722779    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e3ef760-19e1-45bc-b806-57d79f5082db-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:51.722856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.722793    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfkk9\" (UniqueName: \"kubernetes.io/projected/1e3ef760-19e1-45bc-b806-57d79f5082db-kube-api-access-pfkk9\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:51.739371 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.739352    4524 scope.go:122] "RemoveContainer" containerID="3dcd0747cf48ef6b9a8c3a58428d58adc28f9995be16d70bc4709ade02733049"
Oct 30 19:47:51.739643 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:47:51.739622    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dcd0747cf48ef6b9a8c3a58428d58adc28f9995be16d70bc4709ade02733049\": container with ID starting with 3dcd0747cf48ef6b9a8c3a58428d58adc28f9995be16d70bc4709ade02733049 not found: ID does not exist" containerID="3dcd0747cf48ef6b9a8c3a58428d58adc28f9995be16d70bc4709ade02733049"
Oct 30 19:47:51.739718 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.739665    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dcd0747cf48ef6b9a8c3a58428d58adc28f9995be16d70bc4709ade02733049"} err="failed to get container status \"3dcd0747cf48ef6b9a8c3a58428d58adc28f9995be16d70bc4709ade02733049\": rpc error: code = NotFound desc = could not find container \"3dcd0747cf48ef6b9a8c3a58428d58adc28f9995be16d70bc4709ade02733049\": container with ID starting with 3dcd0747cf48ef6b9a8c3a58428d58adc28f9995be16d70bc4709ade02733049 not found: ID does not exist"
Oct 30 19:47:51.748178 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.748139    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"]
Oct 30 19:47:51.754049 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:51.754021    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-c2b092-openshift-default-758cf88db7-wd7j4"]
Oct 30 19:47:52.395624 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:52.395593    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e3ef760-19e1-45bc-b806-57d79f5082db" path="/var/lib/kubelet/pods/1e3ef760-19e1-45bc-b806-57d79f5082db/volumes"
Oct 30 19:47:52.396146 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:52.396127    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="944c40c9-95ff-419a-8429-2e8c9a2eefba" path="/var/lib/kubelet/pods/944c40c9-95ff-419a-8429-2e8c9a2eefba/volumes"
Oct 30 19:47:57.183598 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.183555    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"]
Oct 30 19:47:57.184140 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.183813    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8" podUID="c3f46492-0c06-4e9d-bd74-793bee10103c" containerName="maas-api" containerID="cri-o://66f6bf2a143ef966535dcd51d085f684ee1fa62e502c3ee22fae54203d55d3dd" gracePeriod=30
Oct 30 19:47:57.291558 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.291521    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"]
Oct 30 19:47:57.292026 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.291990    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k" podUID="eadb3435-17d0-4041-8210-3b5cfe7d50bc" containerName="maas-api" containerID="cri-o://5cc0bfbb90853618f2fc33c117a7ad1ffc8a00a03d01069aa58da6683d4e26cc" gracePeriod=30
Oct 30 19:47:57.586388 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.586363    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"
Oct 30 19:47:57.631611 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.631590    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"
Oct 30 19:47:57.740250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.735611    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c3f46492-0c06-4e9d-bd74-793bee10103c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3f46492-0c06-4e9d-bd74-793bee10103c-openshift-service-ca\") pod \"c3f46492-0c06-4e9d-bd74-793bee10103c\" (UID: \"c3f46492-0c06-4e9d-bd74-793bee10103c\") "
Oct 30 19:47:57.740250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.735674    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3f46492-0c06-4e9d-bd74-793bee10103c-kube-api-access-nnnn9\" (UniqueName: \"kubernetes.io/projected/c3f46492-0c06-4e9d-bd74-793bee10103c-kube-api-access-nnnn9\") pod \"c3f46492-0c06-4e9d-bd74-793bee10103c\" (UID: \"c3f46492-0c06-4e9d-bd74-793bee10103c\") "
Oct 30 19:47:57.740250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.735718    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eadb3435-17d0-4041-8210-3b5cfe7d50bc-kube-api-access-6fkkw\" (UniqueName: \"kubernetes.io/projected/eadb3435-17d0-4041-8210-3b5cfe7d50bc-kube-api-access-6fkkw\") pod \"eadb3435-17d0-4041-8210-3b5cfe7d50bc\" (UID: \"eadb3435-17d0-4041-8210-3b5cfe7d50bc\") "
Oct 30 19:47:57.740250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.735932    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c3f46492-0c06-4e9d-bd74-793bee10103c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3f46492-0c06-4e9d-bd74-793bee10103c-metrics-certs\") pod \"c3f46492-0c06-4e9d-bd74-793bee10103c\" (UID: \"c3f46492-0c06-4e9d-bd74-793bee10103c\") "
Oct 30 19:47:57.740250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.735961    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eadb3435-17d0-4041-8210-3b5cfe7d50bc-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eadb3435-17d0-4041-8210-3b5cfe7d50bc-openshift-service-ca\") pod \"eadb3435-17d0-4041-8210-3b5cfe7d50bc\" (UID: \"eadb3435-17d0-4041-8210-3b5cfe7d50bc\") "
Oct 30 19:47:57.740250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.735995    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eadb3435-17d0-4041-8210-3b5cfe7d50bc-metrics-certs\" (UniqueName: \"kubernetes.io/secret/eadb3435-17d0-4041-8210-3b5cfe7d50bc-metrics-certs\") pod \"eadb3435-17d0-4041-8210-3b5cfe7d50bc\" (UID: \"eadb3435-17d0-4041-8210-3b5cfe7d50bc\") "
Oct 30 19:47:57.740250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.736033    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eadb3435-17d0-4041-8210-3b5cfe7d50bc-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eadb3435-17d0-4041-8210-3b5cfe7d50bc-maas-api-tls\") pod \"eadb3435-17d0-4041-8210-3b5cfe7d50bc\" (UID: \"eadb3435-17d0-4041-8210-3b5cfe7d50bc\") "
Oct 30 19:47:57.740250 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.736078    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c3f46492-0c06-4e9d-bd74-793bee10103c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3f46492-0c06-4e9d-bd74-793bee10103c-maas-api-tls\") pod \"c3f46492-0c06-4e9d-bd74-793bee10103c\" (UID: \"c3f46492-0c06-4e9d-bd74-793bee10103c\") "
Oct 30 19:47:57.744738 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.744394    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eadb3435-17d0-4041-8210-3b5cfe7d50bc-openshift-service-ca" pod "eadb3435-17d0-4041-8210-3b5cfe7d50bc" (UID: "eadb3435-17d0-4041-8210-3b5cfe7d50bc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:57.752064 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.751807    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3f46492-0c06-4e9d-bd74-793bee10103c-openshift-service-ca" pod "c3f46492-0c06-4e9d-bd74-793bee10103c" (UID: "c3f46492-0c06-4e9d-bd74-793bee10103c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:57.757718 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.757687    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eadb3435-17d0-4041-8210-3b5cfe7d50bc-maas-api-tls" pod "eadb3435-17d0-4041-8210-3b5cfe7d50bc" (UID: "eadb3435-17d0-4041-8210-3b5cfe7d50bc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:57.758697 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.758654    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3f46492-0c06-4e9d-bd74-793bee10103c-kube-api-access-nnnn9" pod "c3f46492-0c06-4e9d-bd74-793bee10103c" (UID: "c3f46492-0c06-4e9d-bd74-793bee10103c"). InnerVolumeSpecName "kube-api-access-nnnn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:57.761355 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.761308    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eadb3435-17d0-4041-8210-3b5cfe7d50bc-kube-api-access-6fkkw" pod "eadb3435-17d0-4041-8210-3b5cfe7d50bc" (UID: "eadb3435-17d0-4041-8210-3b5cfe7d50bc"). InnerVolumeSpecName "kube-api-access-6fkkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:57.767106 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.767043    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3f46492-0c06-4e9d-bd74-793bee10103c-maas-api-tls" pod "c3f46492-0c06-4e9d-bd74-793bee10103c" (UID: "c3f46492-0c06-4e9d-bd74-793bee10103c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:57.843262 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.836373    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eadb3435-17d0-4041-8210-3b5cfe7d50bc-metrics-certs" pod "eadb3435-17d0-4041-8210-3b5cfe7d50bc" (UID: "eadb3435-17d0-4041-8210-3b5cfe7d50bc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:57.847361 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.847323    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3f46492-0c06-4e9d-bd74-793bee10103c-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:57.847361 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.847365    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nnnn9\" (UniqueName: \"kubernetes.io/projected/c3f46492-0c06-4e9d-bd74-793bee10103c-kube-api-access-nnnn9\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:57.847602 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.847382    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fkkw\" (UniqueName: \"kubernetes.io/projected/eadb3435-17d0-4041-8210-3b5cfe7d50bc-kube-api-access-6fkkw\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:57.847602 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.847397    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eadb3435-17d0-4041-8210-3b5cfe7d50bc-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:57.847602 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.847412    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eadb3435-17d0-4041-8210-3b5cfe7d50bc-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:57.847602 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.847429    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eadb3435-17d0-4041-8210-3b5cfe7d50bc-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:57.847602 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.847447    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3f46492-0c06-4e9d-bd74-793bee10103c-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:57.853564 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.852721    4524 generic.go:356] "Generic (PLEG): container finished" podID="c3f46492-0c06-4e9d-bd74-793bee10103c" containerID="66f6bf2a143ef966535dcd51d085f684ee1fa62e502c3ee22fae54203d55d3dd" exitCode=0
Oct 30 19:47:57.853564 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.852884    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8" event={"ID":"c3f46492-0c06-4e9d-bd74-793bee10103c","Type":"ContainerDied","Data":"66f6bf2a143ef966535dcd51d085f684ee1fa62e502c3ee22fae54203d55d3dd"}
Oct 30 19:47:57.853564 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.852935    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8" event={"ID":"c3f46492-0c06-4e9d-bd74-793bee10103c","Type":"ContainerDied","Data":"c86e697bc22baf5c4ecd8d3881b7e524c62c3a6e621a8b03b081368af2cd9a22"}
Oct 30 19:47:57.853564 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.852966    4524 scope.go:122] "RemoveContainer" containerID="66f6bf2a143ef966535dcd51d085f684ee1fa62e502c3ee22fae54203d55d3dd"
Oct 30 19:47:57.853564 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.853110    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"
Oct 30 19:47:57.859275 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.859239    4524 generic.go:356] "Generic (PLEG): container finished" podID="eadb3435-17d0-4041-8210-3b5cfe7d50bc" containerID="5cc0bfbb90853618f2fc33c117a7ad1ffc8a00a03d01069aa58da6683d4e26cc" exitCode=0
Oct 30 19:47:57.859402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.859372    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"
Oct 30 19:47:57.859469 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.859450    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k" event={"ID":"eadb3435-17d0-4041-8210-3b5cfe7d50bc","Type":"ContainerDied","Data":"5cc0bfbb90853618f2fc33c117a7ad1ffc8a00a03d01069aa58da6683d4e26cc"}
Oct 30 19:47:57.859590 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.859573    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k" event={"ID":"eadb3435-17d0-4041-8210-3b5cfe7d50bc","Type":"ContainerDied","Data":"7a8d4ac39d73b91755ab2adbb1605fdd8ba26c29b31aad1082eb16eadc42d0ed"}
Oct 30 19:47:57.861386 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.861358    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3f46492-0c06-4e9d-bd74-793bee10103c-metrics-certs" pod "c3f46492-0c06-4e9d-bd74-793bee10103c" (UID: "c3f46492-0c06-4e9d-bd74-793bee10103c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:57.893263 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.893232    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"]
Oct 30 19:47:57.895401 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.895271    4524 scope.go:122] "RemoveContainer" containerID="66f6bf2a143ef966535dcd51d085f684ee1fa62e502c3ee22fae54203d55d3dd"
Oct 30 19:47:57.895715 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:47:57.895685    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66f6bf2a143ef966535dcd51d085f684ee1fa62e502c3ee22fae54203d55d3dd\": container with ID starting with 66f6bf2a143ef966535dcd51d085f684ee1fa62e502c3ee22fae54203d55d3dd not found: ID does not exist" containerID="66f6bf2a143ef966535dcd51d085f684ee1fa62e502c3ee22fae54203d55d3dd"
Oct 30 19:47:57.895827 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.895729    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66f6bf2a143ef966535dcd51d085f684ee1fa62e502c3ee22fae54203d55d3dd"} err="failed to get container status \"66f6bf2a143ef966535dcd51d085f684ee1fa62e502c3ee22fae54203d55d3dd\": rpc error: code = NotFound desc = could not find container \"66f6bf2a143ef966535dcd51d085f684ee1fa62e502c3ee22fae54203d55d3dd\": container with ID starting with 66f6bf2a143ef966535dcd51d085f684ee1fa62e502c3ee22fae54203d55d3dd not found: ID does not exist"
Oct 30 19:47:57.895827 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.895762    4524 scope.go:122] "RemoveContainer" containerID="5cc0bfbb90853618f2fc33c117a7ad1ffc8a00a03d01069aa58da6683d4e26cc"
Oct 30 19:47:57.897316 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.897293    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c0a27e-56c7bf9cb6-44j7k"]
Oct 30 19:47:57.909118 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.909080    4524 scope.go:122] "RemoveContainer" containerID="5cc0bfbb90853618f2fc33c117a7ad1ffc8a00a03d01069aa58da6683d4e26cc"
Oct 30 19:47:57.909468 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:47:57.909421    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cc0bfbb90853618f2fc33c117a7ad1ffc8a00a03d01069aa58da6683d4e26cc\": container with ID starting with 5cc0bfbb90853618f2fc33c117a7ad1ffc8a00a03d01069aa58da6683d4e26cc not found: ID does not exist" containerID="5cc0bfbb90853618f2fc33c117a7ad1ffc8a00a03d01069aa58da6683d4e26cc"
Oct 30 19:47:57.909468 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.909447    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cc0bfbb90853618f2fc33c117a7ad1ffc8a00a03d01069aa58da6683d4e26cc"} err="failed to get container status \"5cc0bfbb90853618f2fc33c117a7ad1ffc8a00a03d01069aa58da6683d4e26cc\": rpc error: code = NotFound desc = could not find container \"5cc0bfbb90853618f2fc33c117a7ad1ffc8a00a03d01069aa58da6683d4e26cc\": container with ID starting with 5cc0bfbb90853618f2fc33c117a7ad1ffc8a00a03d01069aa58da6683d4e26cc not found: ID does not exist"
Oct 30 19:47:57.949602 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:57.949567    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c3f46492-0c06-4e9d-bd74-793bee10103c-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:58.183440 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:58.183410    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"]
Oct 30 19:47:58.194429 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:58.194398    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-1b1531-75549f7645-s95z8"]
Oct 30 19:47:58.407214 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:58.405246    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c3f46492-0c06-4e9d-bd74-793bee10103c" path="/var/lib/kubelet/pods/c3f46492-0c06-4e9d-bd74-793bee10103c/volumes"
Oct 30 19:47:58.407214 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:58.405866    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eadb3435-17d0-4041-8210-3b5cfe7d50bc" path="/var/lib/kubelet/pods/eadb3435-17d0-4041-8210-3b5cfe7d50bc/volumes"
Oct 30 19:47:58.799049 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:58.799014    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"]
Oct 30 19:47:58.799294 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:58.799265    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj" podUID="662686fd-7d04-48b9-bf7e-adbd30299109" containerName="maas-api" containerID="cri-o://9d80567b1d0a19ce86df03b37e9753f2599fde6037e6e70d51d5bb73ad5f13e5" gracePeriod=30
Oct 30 19:47:59.173248 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.169539    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"
Oct 30 19:47:59.269962 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.269928    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/662686fd-7d04-48b9-bf7e-adbd30299109-metrics-certs\" (UniqueName: \"kubernetes.io/secret/662686fd-7d04-48b9-bf7e-adbd30299109-metrics-certs\") pod \"662686fd-7d04-48b9-bf7e-adbd30299109\" (UID: \"662686fd-7d04-48b9-bf7e-adbd30299109\") "
Oct 30 19:47:59.270498 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.270010    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/662686fd-7d04-48b9-bf7e-adbd30299109-kube-api-access-5llwr\" (UniqueName: \"kubernetes.io/projected/662686fd-7d04-48b9-bf7e-adbd30299109-kube-api-access-5llwr\") pod \"662686fd-7d04-48b9-bf7e-adbd30299109\" (UID: \"662686fd-7d04-48b9-bf7e-adbd30299109\") "
Oct 30 19:47:59.270498 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.270093    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/662686fd-7d04-48b9-bf7e-adbd30299109-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/662686fd-7d04-48b9-bf7e-adbd30299109-openshift-service-ca\") pod \"662686fd-7d04-48b9-bf7e-adbd30299109\" (UID: \"662686fd-7d04-48b9-bf7e-adbd30299109\") "
Oct 30 19:47:59.270498 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.270212    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/662686fd-7d04-48b9-bf7e-adbd30299109-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/662686fd-7d04-48b9-bf7e-adbd30299109-maas-api-tls\") pod \"662686fd-7d04-48b9-bf7e-adbd30299109\" (UID: \"662686fd-7d04-48b9-bf7e-adbd30299109\") "
Oct 30 19:47:59.271153 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.271116    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/662686fd-7d04-48b9-bf7e-adbd30299109-openshift-service-ca" pod "662686fd-7d04-48b9-bf7e-adbd30299109" (UID: "662686fd-7d04-48b9-bf7e-adbd30299109"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:59.273756 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.273702    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/662686fd-7d04-48b9-bf7e-adbd30299109-metrics-certs" pod "662686fd-7d04-48b9-bf7e-adbd30299109" (UID: "662686fd-7d04-48b9-bf7e-adbd30299109"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:59.274863 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.274844    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/662686fd-7d04-48b9-bf7e-adbd30299109-kube-api-access-5llwr" pod "662686fd-7d04-48b9-bf7e-adbd30299109" (UID: "662686fd-7d04-48b9-bf7e-adbd30299109"). InnerVolumeSpecName "kube-api-access-5llwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:59.275328 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.275302    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/662686fd-7d04-48b9-bf7e-adbd30299109-maas-api-tls" pod "662686fd-7d04-48b9-bf7e-adbd30299109" (UID: "662686fd-7d04-48b9-bf7e-adbd30299109"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:47:59.371652 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.371617    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/662686fd-7d04-48b9-bf7e-adbd30299109-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:59.371652 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.371654    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/662686fd-7d04-48b9-bf7e-adbd30299109-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:59.371909 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.371672    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5llwr\" (UniqueName: \"kubernetes.io/projected/662686fd-7d04-48b9-bf7e-adbd30299109-kube-api-access-5llwr\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:59.371909 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.371687    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/662686fd-7d04-48b9-bf7e-adbd30299109-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:59.886839 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.886793    4524 generic.go:356] "Generic (PLEG): container finished" podID="662686fd-7d04-48b9-bf7e-adbd30299109" containerID="9d80567b1d0a19ce86df03b37e9753f2599fde6037e6e70d51d5bb73ad5f13e5" exitCode=0
Oct 30 19:47:59.887019 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.886966    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj" event={"ID":"662686fd-7d04-48b9-bf7e-adbd30299109","Type":"ContainerDied","Data":"9d80567b1d0a19ce86df03b37e9753f2599fde6037e6e70d51d5bb73ad5f13e5"}
Oct 30 19:47:59.887019 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.887008    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj" event={"ID":"662686fd-7d04-48b9-bf7e-adbd30299109","Type":"ContainerDied","Data":"98dd9c806afb962b1c5cead0a11ab375f1f2e70eb8d2ab1d971a7dcef1526a19"}
Oct 30 19:47:59.887119 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.887040    4524 scope.go:122] "RemoveContainer" containerID="9d80567b1d0a19ce86df03b37e9753f2599fde6037e6e70d51d5bb73ad5f13e5"
Oct 30 19:47:59.887154 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.887134    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"
Oct 30 19:47:59.908979 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.908960    4524 scope.go:122] "RemoveContainer" containerID="9d80567b1d0a19ce86df03b37e9753f2599fde6037e6e70d51d5bb73ad5f13e5"
Oct 30 19:47:59.909352 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:47:59.909322    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d80567b1d0a19ce86df03b37e9753f2599fde6037e6e70d51d5bb73ad5f13e5\": container with ID starting with 9d80567b1d0a19ce86df03b37e9753f2599fde6037e6e70d51d5bb73ad5f13e5 not found: ID does not exist" containerID="9d80567b1d0a19ce86df03b37e9753f2599fde6037e6e70d51d5bb73ad5f13e5"
Oct 30 19:47:59.909440 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.909361    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d80567b1d0a19ce86df03b37e9753f2599fde6037e6e70d51d5bb73ad5f13e5"} err="failed to get container status \"9d80567b1d0a19ce86df03b37e9753f2599fde6037e6e70d51d5bb73ad5f13e5\": rpc error: code = NotFound desc = could not find container \"9d80567b1d0a19ce86df03b37e9753f2599fde6037e6e70d51d5bb73ad5f13e5\": container with ID starting with 9d80567b1d0a19ce86df03b37e9753f2599fde6037e6e70d51d5bb73ad5f13e5 not found: ID does not exist"
Oct 30 19:47:59.924191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.919454    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"]
Oct 30 19:47:59.924728 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:47:59.924701    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-826518-7f8957fc9-9nhzj"]
Oct 30 19:48:00.432231 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:00.431214    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="662686fd-7d04-48b9-bf7e-adbd30299109" path="/var/lib/kubelet/pods/662686fd-7d04-48b9-bf7e-adbd30299109/volumes"
Oct 30 19:48:05.912191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:05.912089    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"]
Oct 30 19:48:05.912653 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:05.912516    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt" podUID="29b2b2ca-a677-4d48-ab5c-9b202e007103" containerName="istio-proxy" containerID="cri-o://454e70ec4c1b35565716d90254df05b5e7048c6c94ad9f417f14bb24aaf28837" gracePeriod=30
Oct 30 19:48:05.928618 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:05.927558    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"]
Oct 30 19:48:05.928618 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:05.927850    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5" podUID="caf02c37-523d-43ba-b171-4bcf692ed088" containerName="istio-proxy" containerID="cri-o://0f8e67d6ad3da72cca4fce37ffa54b98fa2f9130df826dc61b5ec294f0f732a6" gracePeriod=30
Oct 30 19:48:07.161716 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:07.161682    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"]
Oct 30 19:48:07.162264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:07.161905    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km" podUID="0d49fd92-802a-402c-8098-414925be3659" containerName="istio-proxy" containerID="cri-o://37e2a6fa7b944865e186f2fa269fcf61a73a27f009c58cd30faaab6a2b52395e" gracePeriod=30
Oct 30 19:48:11.183932 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.183666    4524 generic.go:356] "Generic (PLEG): container finished" podID="caf02c37-523d-43ba-b171-4bcf692ed088" containerID="0f8e67d6ad3da72cca4fce37ffa54b98fa2f9130df826dc61b5ec294f0f732a6" exitCode=0
Oct 30 19:48:11.183932 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.183838    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5" event={"ID":"caf02c37-523d-43ba-b171-4bcf692ed088","Type":"ContainerDied","Data":"0f8e67d6ad3da72cca4fce37ffa54b98fa2f9130df826dc61b5ec294f0f732a6"}
Oct 30 19:48:11.230922 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.228677    4524 generic.go:356] "Generic (PLEG): container finished" podID="29b2b2ca-a677-4d48-ab5c-9b202e007103" containerID="454e70ec4c1b35565716d90254df05b5e7048c6c94ad9f417f14bb24aaf28837" exitCode=0
Oct 30 19:48:11.230922 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.228818    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt" event={"ID":"29b2b2ca-a677-4d48-ab5c-9b202e007103","Type":"ContainerDied","Data":"454e70ec4c1b35565716d90254df05b5e7048c6c94ad9f417f14bb24aaf28837"}
Oct 30 19:48:11.381425 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.381050    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:48:11.384247 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.384223    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:48:11.435956 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.435926    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29b2b2ca-a677-4d48-ab5c-9b202e007103-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29b2b2ca-a677-4d48-ab5c-9b202e007103-istiod-ca-cert\") pod \"29b2b2ca-a677-4d48-ab5c-9b202e007103\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") "
Oct 30 19:48:11.436118 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.435989    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-workload-certs\") pod \"29b2b2ca-a677-4d48-ab5c-9b202e007103\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") "
Oct 30 19:48:11.436118 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436099    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-workload-socket\") pod \"29b2b2ca-a677-4d48-ab5c-9b202e007103\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") "
Oct 30 19:48:11.436246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436138    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-token\" (UniqueName: \"kubernetes.io/projected/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-token\") pod \"29b2b2ca-a677-4d48-ab5c-9b202e007103\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") "
Oct 30 19:48:11.436246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436190    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-workload-certs\") pod \"caf02c37-523d-43ba-b171-4bcf692ed088\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") "
Oct 30 19:48:11.436246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436220    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/caf02c37-523d-43ba-b171-4bcf692ed088-kube-api-access-svmff\" (UniqueName: \"kubernetes.io/projected/caf02c37-523d-43ba-b171-4bcf692ed088-kube-api-access-svmff\") pod \"caf02c37-523d-43ba-b171-4bcf692ed088\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") "
Oct 30 19:48:11.436416 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436268    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-credential-socket\") pod \"caf02c37-523d-43ba-b171-4bcf692ed088\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") "
Oct 30 19:48:11.436416 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436297    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29b2b2ca-a677-4d48-ab5c-9b202e007103-kube-api-access-sbdvr\" (UniqueName: \"kubernetes.io/projected/29b2b2ca-a677-4d48-ab5c-9b202e007103-kube-api-access-sbdvr\") pod \"29b2b2ca-a677-4d48-ab5c-9b202e007103\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") "
Oct 30 19:48:11.436416 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436322    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-istio-data\") pod \"caf02c37-523d-43ba-b171-4bcf692ed088\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") "
Oct 30 19:48:11.436416 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436327    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-workload-certs" pod "29b2b2ca-a677-4d48-ab5c-9b202e007103" (UID: "29b2b2ca-a677-4d48-ab5c-9b202e007103"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:11.436416 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436396    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-envoy\") pod \"29b2b2ca-a677-4d48-ab5c-9b202e007103\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") "
Oct 30 19:48:11.436685 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436419    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-workload-socket" pod "29b2b2ca-a677-4d48-ab5c-9b202e007103" (UID: "29b2b2ca-a677-4d48-ab5c-9b202e007103"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:11.436685 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436433    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-podinfo\") pod \"29b2b2ca-a677-4d48-ab5c-9b202e007103\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") "
Oct 30 19:48:11.436685 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436453    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29b2b2ca-a677-4d48-ab5c-9b202e007103-istiod-ca-cert" pod "29b2b2ca-a677-4d48-ab5c-9b202e007103" (UID: "29b2b2ca-a677-4d48-ab5c-9b202e007103"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:11.436685 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436542    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-workload-certs" pod "caf02c37-523d-43ba-b171-4bcf692ed088" (UID: "caf02c37-523d-43ba-b171-4bcf692ed088"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:11.436685 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436475    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-data\") pod \"29b2b2ca-a677-4d48-ab5c-9b202e007103\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") "
Oct 30 19:48:11.436685 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436665    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-istio-envoy\") pod \"caf02c37-523d-43ba-b171-4bcf692ed088\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") "
Oct 30 19:48:11.437003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436688    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-data" pod "29b2b2ca-a677-4d48-ab5c-9b202e007103" (UID: "29b2b2ca-a677-4d48-ab5c-9b202e007103"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:11.437003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436694    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-istio-data" pod "caf02c37-523d-43ba-b171-4bcf692ed088" (UID: "caf02c37-523d-43ba-b171-4bcf692ed088"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:11.437003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436712    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-credential-socket\") pod \"29b2b2ca-a677-4d48-ab5c-9b202e007103\" (UID: \"29b2b2ca-a677-4d48-ab5c-9b202e007103\") "
Oct 30 19:48:11.437003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436766    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-credential-socket" pod "caf02c37-523d-43ba-b171-4bcf692ed088" (UID: "caf02c37-523d-43ba-b171-4bcf692ed088"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:11.437003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436781    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-workload-socket\") pod \"caf02c37-523d-43ba-b171-4bcf692ed088\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") "
Oct 30 19:48:11.437003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436823    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/caf02c37-523d-43ba-b171-4bcf692ed088-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caf02c37-523d-43ba-b171-4bcf692ed088-istio-podinfo\") pod \"caf02c37-523d-43ba-b171-4bcf692ed088\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") "
Oct 30 19:48:11.437003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436894    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/caf02c37-523d-43ba-b171-4bcf692ed088-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caf02c37-523d-43ba-b171-4bcf692ed088-istiod-ca-cert\") pod \"caf02c37-523d-43ba-b171-4bcf692ed088\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") "
Oct 30 19:48:11.437003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436927    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/caf02c37-523d-43ba-b171-4bcf692ed088-istio-token\" (UniqueName: \"kubernetes.io/projected/caf02c37-523d-43ba-b171-4bcf692ed088-istio-token\") pod \"caf02c37-523d-43ba-b171-4bcf692ed088\" (UID: \"caf02c37-523d-43ba-b171-4bcf692ed088\") "
Oct 30 19:48:11.437003 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.436926    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-credential-socket" pod "29b2b2ca-a677-4d48-ab5c-9b202e007103" (UID: "29b2b2ca-a677-4d48-ab5c-9b202e007103"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:11.437518 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.437489    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:11.437589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.437517    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29b2b2ca-a677-4d48-ab5c-9b202e007103-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:11.437589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.437532    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:11.437589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.437546    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:11.437589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.437561    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:11.437589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.437575    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:11.437848 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.437589    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:11.437848 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.437602    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:11.438489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.438214    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caf02c37-523d-43ba-b171-4bcf692ed088-istiod-ca-cert" pod "caf02c37-523d-43ba-b171-4bcf692ed088" (UID: "caf02c37-523d-43ba-b171-4bcf692ed088"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:11.439952 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.439896    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-workload-socket" pod "caf02c37-523d-43ba-b171-4bcf692ed088" (UID: "caf02c37-523d-43ba-b171-4bcf692ed088"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:11.443955 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.443863    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-envoy" pod "29b2b2ca-a677-4d48-ab5c-9b202e007103" (UID: "29b2b2ca-a677-4d48-ab5c-9b202e007103"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:11.443955 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.443932    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/caf02c37-523d-43ba-b171-4bcf692ed088-istio-podinfo" pod "caf02c37-523d-43ba-b171-4bcf692ed088" (UID: "caf02c37-523d-43ba-b171-4bcf692ed088"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:48:11.444927 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.444113    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29b2b2ca-a677-4d48-ab5c-9b202e007103-kube-api-access-sbdvr" pod "29b2b2ca-a677-4d48-ab5c-9b202e007103" (UID: "29b2b2ca-a677-4d48-ab5c-9b202e007103"). InnerVolumeSpecName "kube-api-access-sbdvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:11.444927 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.444221    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caf02c37-523d-43ba-b171-4bcf692ed088-kube-api-access-svmff" pod "caf02c37-523d-43ba-b171-4bcf692ed088" (UID: "caf02c37-523d-43ba-b171-4bcf692ed088"). InnerVolumeSpecName "kube-api-access-svmff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:11.444927 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.444520    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-podinfo" pod "29b2b2ca-a677-4d48-ab5c-9b202e007103" (UID: "29b2b2ca-a677-4d48-ab5c-9b202e007103"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:48:11.444927 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.444881    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-istio-envoy" pod "caf02c37-523d-43ba-b171-4bcf692ed088" (UID: "caf02c37-523d-43ba-b171-4bcf692ed088"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:11.445944 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.445537    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caf02c37-523d-43ba-b171-4bcf692ed088-istio-token" pod "caf02c37-523d-43ba-b171-4bcf692ed088" (UID: "caf02c37-523d-43ba-b171-4bcf692ed088"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:11.445944 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.445650    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-token" pod "29b2b2ca-a677-4d48-ab5c-9b202e007103" (UID: "29b2b2ca-a677-4d48-ab5c-9b202e007103"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:11.539081 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.539005    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:11.539081 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.539037    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:11.539081 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.539049    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:11.539081 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.539057    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf02c37-523d-43ba-b171-4bcf692ed088-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:11.539081 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.539066    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caf02c37-523d-43ba-b171-4bcf692ed088-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:11.539081 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.539075    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caf02c37-523d-43ba-b171-4bcf692ed088-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:11.539081 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.539084    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caf02c37-523d-43ba-b171-4bcf692ed088-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:11.539489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.539092    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29b2b2ca-a677-4d48-ab5c-9b202e007103-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:11.539489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.539100    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svmff\" (UniqueName: \"kubernetes.io/projected/caf02c37-523d-43ba-b171-4bcf692ed088-kube-api-access-svmff\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:11.539489 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:11.539114    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbdvr\" (UniqueName: \"kubernetes.io/projected/29b2b2ca-a677-4d48-ab5c-9b202e007103-kube-api-access-sbdvr\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:12.236647 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.236593    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"
Oct 30 19:48:12.236647 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.236630    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5" event={"ID":"caf02c37-523d-43ba-b171-4bcf692ed088","Type":"ContainerDied","Data":"2418af1dde3b897fb45f03bff3cae3139e3f9e07cf079aa5e58db2e664e50058"}
Oct 30 19:48:12.237077 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.236692    4524 scope.go:122] "RemoveContainer" containerID="0f8e67d6ad3da72cca4fce37ffa54b98fa2f9130df826dc61b5ec294f0f732a6"
Oct 30 19:48:12.238199 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.238146    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt" event={"ID":"29b2b2ca-a677-4d48-ab5c-9b202e007103","Type":"ContainerDied","Data":"c7cd65618c13c7eae6e5ed6128fb8a2e9d8b615177c0a4059757a862a6b96487"}
Oct 30 19:48:12.238280 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.238214    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"
Oct 30 19:48:12.266686 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.266666    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"]
Oct 30 19:48:12.274161 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.274134    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-tktnt"]
Oct 30 19:48:12.277093 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.277069    4524 scope.go:122] "RemoveContainer" containerID="454e70ec4c1b35565716d90254df05b5e7048c6c94ad9f417f14bb24aaf28837"
Oct 30 19:48:12.286157 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.286123    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"]
Oct 30 19:48:12.291557 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.291535    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w4-1b1531-openshift-default-56df9dbc55-fblb5"]
Oct 30 19:48:12.396251 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.395382    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29b2b2ca-a677-4d48-ab5c-9b202e007103" path="/var/lib/kubelet/pods/29b2b2ca-a677-4d48-ab5c-9b202e007103/volumes"
Oct 30 19:48:12.396251 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.395896    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="caf02c37-523d-43ba-b171-4bcf692ed088" path="/var/lib/kubelet/pods/caf02c37-523d-43ba-b171-4bcf692ed088/volumes"
Oct 30 19:48:12.449128 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.449097    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:48:12.560601 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.558980    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d49fd92-802a-402c-8098-414925be3659-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d49fd92-802a-402c-8098-414925be3659-istiod-ca-cert\") pod \"0d49fd92-802a-402c-8098-414925be3659\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") "
Oct 30 19:48:12.560601 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.559033    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-istio-envoy\") pod \"0d49fd92-802a-402c-8098-414925be3659\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") "
Oct 30 19:48:12.560601 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.559069    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0d49fd92-802a-402c-8098-414925be3659-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d49fd92-802a-402c-8098-414925be3659-istio-podinfo\") pod \"0d49fd92-802a-402c-8098-414925be3659\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") "
Oct 30 19:48:12.560601 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.559123    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-workload-certs\") pod \"0d49fd92-802a-402c-8098-414925be3659\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") "
Oct 30 19:48:12.560601 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.559189    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-istio-data\") pod \"0d49fd92-802a-402c-8098-414925be3659\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") "
Oct 30 19:48:12.560601 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.559234    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d49fd92-802a-402c-8098-414925be3659-kube-api-access-c2mbh\" (UniqueName: \"kubernetes.io/projected/0d49fd92-802a-402c-8098-414925be3659-kube-api-access-c2mbh\") pod \"0d49fd92-802a-402c-8098-414925be3659\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") "
Oct 30 19:48:12.560601 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.559277    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d49fd92-802a-402c-8098-414925be3659-istio-token\" (UniqueName: \"kubernetes.io/projected/0d49fd92-802a-402c-8098-414925be3659-istio-token\") pod \"0d49fd92-802a-402c-8098-414925be3659\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") "
Oct 30 19:48:12.560601 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.559337    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-credential-socket\") pod \"0d49fd92-802a-402c-8098-414925be3659\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") "
Oct 30 19:48:12.560601 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.559751    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-workload-certs" pod "0d49fd92-802a-402c-8098-414925be3659" (UID: "0d49fd92-802a-402c-8098-414925be3659"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:12.562695 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.562324    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-credential-socket" pod "0d49fd92-802a-402c-8098-414925be3659" (UID: "0d49fd92-802a-402c-8098-414925be3659"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:12.562902 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.562880    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-istio-data" pod "0d49fd92-802a-402c-8098-414925be3659" (UID: "0d49fd92-802a-402c-8098-414925be3659"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:12.563561 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.563526    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-workload-socket" pod "0d49fd92-802a-402c-8098-414925be3659" (UID: "0d49fd92-802a-402c-8098-414925be3659"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:12.569984 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.569935    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d49fd92-802a-402c-8098-414925be3659-istio-token" pod "0d49fd92-802a-402c-8098-414925be3659" (UID: "0d49fd92-802a-402c-8098-414925be3659"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:12.572147 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.572110    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0d49fd92-802a-402c-8098-414925be3659-istio-podinfo" pod "0d49fd92-802a-402c-8098-414925be3659" (UID: "0d49fd92-802a-402c-8098-414925be3659"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:48:12.572407 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.572332    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-istio-envoy" pod "0d49fd92-802a-402c-8098-414925be3659" (UID: "0d49fd92-802a-402c-8098-414925be3659"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:12.585915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.578317    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-workload-socket\") pod \"0d49fd92-802a-402c-8098-414925be3659\" (UID: \"0d49fd92-802a-402c-8098-414925be3659\") "
Oct 30 19:48:12.585915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.578876    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:12.585915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.578900    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:12.585915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.578915    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:12.585915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.578928    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d49fd92-802a-402c-8098-414925be3659-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:12.585915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.578941    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:12.585915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.578954    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d49fd92-802a-402c-8098-414925be3659-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:12.585915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.578967    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d49fd92-802a-402c-8098-414925be3659-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:12.585915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.581387    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d49fd92-802a-402c-8098-414925be3659-istiod-ca-cert" pod "0d49fd92-802a-402c-8098-414925be3659" (UID: "0d49fd92-802a-402c-8098-414925be3659"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:12.585915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.585843    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d49fd92-802a-402c-8098-414925be3659-kube-api-access-c2mbh" pod "0d49fd92-802a-402c-8098-414925be3659" (UID: "0d49fd92-802a-402c-8098-414925be3659"). InnerVolumeSpecName "kube-api-access-c2mbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:12.637965 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.637939    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"]
Oct 30 19:48:12.680517 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.680484    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d49fd92-802a-402c-8098-414925be3659-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:12.680517 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:12.680510    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2mbh\" (UniqueName: \"kubernetes.io/projected/0d49fd92-802a-402c-8098-414925be3659-kube-api-access-c2mbh\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:13.247894 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:13.247857    4524 generic.go:356] "Generic (PLEG): container finished" podID="0d49fd92-802a-402c-8098-414925be3659" containerID="37e2a6fa7b944865e186f2fa269fcf61a73a27f009c58cd30faaab6a2b52395e" exitCode=0
Oct 30 19:48:13.248487 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:13.247908    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"
Oct 30 19:48:13.248487 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:13.247938    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km" event={"ID":"0d49fd92-802a-402c-8098-414925be3659","Type":"ContainerDied","Data":"37e2a6fa7b944865e186f2fa269fcf61a73a27f009c58cd30faaab6a2b52395e"}
Oct 30 19:48:13.248487 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:13.247990    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km" event={"ID":"0d49fd92-802a-402c-8098-414925be3659","Type":"ContainerDied","Data":"2c435bc51175861f0ebeb67308063a1b9d99ab6bc75e47d00b174f8bf3155eab"}
Oct 30 19:48:13.248487 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:13.248020    4524 scope.go:122] "RemoveContainer" containerID="37e2a6fa7b944865e186f2fa269fcf61a73a27f009c58cd30faaab6a2b52395e"
Oct 30 19:48:13.263196 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:13.263158    4524 scope.go:122] "RemoveContainer" containerID="cb9a95485bf80337be1c2764e89bc5f1000c889a2517d4072846ecd741ba4595"
Oct 30 19:48:13.275549 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:13.275527    4524 scope.go:122] "RemoveContainer" containerID="37e2a6fa7b944865e186f2fa269fcf61a73a27f009c58cd30faaab6a2b52395e"
Oct 30 19:48:13.275945 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:48:13.275914    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37e2a6fa7b944865e186f2fa269fcf61a73a27f009c58cd30faaab6a2b52395e\": container with ID starting with 37e2a6fa7b944865e186f2fa269fcf61a73a27f009c58cd30faaab6a2b52395e not found: ID does not exist" containerID="37e2a6fa7b944865e186f2fa269fcf61a73a27f009c58cd30faaab6a2b52395e"
Oct 30 19:48:13.276047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:13.275956    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37e2a6fa7b944865e186f2fa269fcf61a73a27f009c58cd30faaab6a2b52395e"} err="failed to get container status \"37e2a6fa7b944865e186f2fa269fcf61a73a27f009c58cd30faaab6a2b52395e\": rpc error: code = NotFound desc = could not find container \"37e2a6fa7b944865e186f2fa269fcf61a73a27f009c58cd30faaab6a2b52395e\": container with ID starting with 37e2a6fa7b944865e186f2fa269fcf61a73a27f009c58cd30faaab6a2b52395e not found: ID does not exist"
Oct 30 19:48:13.276047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:13.275980    4524 scope.go:122] "RemoveContainer" containerID="cb9a95485bf80337be1c2764e89bc5f1000c889a2517d4072846ecd741ba4595"
Oct 30 19:48:13.276443 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:48:13.276414    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb9a95485bf80337be1c2764e89bc5f1000c889a2517d4072846ecd741ba4595\": container with ID starting with cb9a95485bf80337be1c2764e89bc5f1000c889a2517d4072846ecd741ba4595 not found: ID does not exist" containerID="cb9a95485bf80337be1c2764e89bc5f1000c889a2517d4072846ecd741ba4595"
Oct 30 19:48:13.276527 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:13.276469    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb9a95485bf80337be1c2764e89bc5f1000c889a2517d4072846ecd741ba4595"} err="failed to get container status \"cb9a95485bf80337be1c2764e89bc5f1000c889a2517d4072846ecd741ba4595\": rpc error: code = NotFound desc = could not find container \"cb9a95485bf80337be1c2764e89bc5f1000c889a2517d4072846ecd741ba4595\": container with ID starting with cb9a95485bf80337be1c2764e89bc5f1000c889a2517d4072846ecd741ba4595 not found: ID does not exist"
Oct 30 19:48:13.278730 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:13.278707    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"]
Oct 30 19:48:13.282027 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:13.282008    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-c0a27e-openshift-default-6d7b6ccf45-z95km"]
Oct 30 19:48:14.399125 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:14.399078    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d49fd92-802a-402c-8098-414925be3659" path="/var/lib/kubelet/pods/0d49fd92-802a-402c-8098-414925be3659/volumes"
Oct 30 19:48:15.625264 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:48:15.625221    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:48:16.686723 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:16.686683    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h"]
Oct 30 19:48:16.697931 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:16.697893    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb"]
Oct 30 19:48:17.946205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:17.946155    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq"]
Oct 30 19:48:17.958042 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:17.958003    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh"]
Oct 30 19:48:21.688261 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:21.688213    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h" podUID="db66fd6a-afe9-47e7-815c-5c66c86081d8" containerName="payload-processing" containerID="cri-o://37569f1588ec4102c1cb7a3ece5b5ac88d7408ec11a574ef809f03d51a2ac79a" gracePeriod=25
Oct 30 19:48:21.698315 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:21.698276    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb" podUID="1affe672-c981-43ad-bd2c-099648e4be4c" containerName="payload-pre-processing" containerID="cri-o://594a7a82458e83016d71647fd6c49048dfdc29051ba1b6e9d86a73e417db102f" gracePeriod=25
Oct 30 19:48:22.504602 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:22.504563    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h" podUID="db66fd6a-afe9-47e7-815c-5c66c86081d8" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:48:22.504893 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:22.504768    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb" podUID="1affe672-c981-43ad-bd2c-099648e4be4c" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:48:22.652075 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:22.652042    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"]
Oct 30 19:48:22.652346 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:22.652319    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2" podUID="8b541712-a45c-41a1-bfcc-fc7349f4355b" containerName="maas-api" containerID="cri-o://8f26347c52826768460bb3361f4ae45f59cbf74d26018b4db32e485883331751" gracePeriod=30
Oct 30 19:48:22.947611 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:22.947567    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq" podUID="7ac2f9eb-a945-483a-9c0a-7ad91b8fd618" containerName="payload-processing" containerID="cri-o://e866f74e712d90cf48a0f818e0cdfdcbee70f49fab1722a1963d019b54907a8c" gracePeriod=25
Oct 30 19:48:22.959164 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:22.959132    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh" podUID="3689eb31-fbba-4cbf-a355-7056ccec100b" containerName="payload-pre-processing" containerID="cri-o://7080939e88c720ab0317423c72842b71ecd72a8f429dfbef10c6fb5341766ff9" gracePeriod=25
Oct 30 19:48:22.971157 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:22.971133    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"
Oct 30 19:48:23.023987 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.023952    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b541712-a45c-41a1-bfcc-fc7349f4355b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b541712-a45c-41a1-bfcc-fc7349f4355b-maas-api-tls\") pod \"8b541712-a45c-41a1-bfcc-fc7349f4355b\" (UID: \"8b541712-a45c-41a1-bfcc-fc7349f4355b\") "
Oct 30 19:48:23.024151 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.024076    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b541712-a45c-41a1-bfcc-fc7349f4355b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b541712-a45c-41a1-bfcc-fc7349f4355b-metrics-certs\") pod \"8b541712-a45c-41a1-bfcc-fc7349f4355b\" (UID: \"8b541712-a45c-41a1-bfcc-fc7349f4355b\") "
Oct 30 19:48:23.024151 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.024117    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b541712-a45c-41a1-bfcc-fc7349f4355b-kube-api-access-84r8n\" (UniqueName: \"kubernetes.io/projected/8b541712-a45c-41a1-bfcc-fc7349f4355b-kube-api-access-84r8n\") pod \"8b541712-a45c-41a1-bfcc-fc7349f4355b\" (UID: \"8b541712-a45c-41a1-bfcc-fc7349f4355b\") "
Oct 30 19:48:23.024395 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.024371    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b541712-a45c-41a1-bfcc-fc7349f4355b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b541712-a45c-41a1-bfcc-fc7349f4355b-openshift-service-ca\") pod \"8b541712-a45c-41a1-bfcc-fc7349f4355b\" (UID: \"8b541712-a45c-41a1-bfcc-fc7349f4355b\") "
Oct 30 19:48:23.024850 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.024823    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b541712-a45c-41a1-bfcc-fc7349f4355b-openshift-service-ca" pod "8b541712-a45c-41a1-bfcc-fc7349f4355b" (UID: "8b541712-a45c-41a1-bfcc-fc7349f4355b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:23.026774 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.026702    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b541712-a45c-41a1-bfcc-fc7349f4355b-kube-api-access-84r8n" pod "8b541712-a45c-41a1-bfcc-fc7349f4355b" (UID: "8b541712-a45c-41a1-bfcc-fc7349f4355b"). InnerVolumeSpecName "kube-api-access-84r8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:23.026880 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.026798    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b541712-a45c-41a1-bfcc-fc7349f4355b-maas-api-tls" pod "8b541712-a45c-41a1-bfcc-fc7349f4355b" (UID: "8b541712-a45c-41a1-bfcc-fc7349f4355b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:48:23.026880 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.026860    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b541712-a45c-41a1-bfcc-fc7349f4355b-metrics-certs" pod "8b541712-a45c-41a1-bfcc-fc7349f4355b" (UID: "8b541712-a45c-41a1-bfcc-fc7349f4355b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:48:23.126544 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.126507    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b541712-a45c-41a1-bfcc-fc7349f4355b-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:23.126544 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.126540    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b541712-a45c-41a1-bfcc-fc7349f4355b-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:23.126544 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.126551    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b541712-a45c-41a1-bfcc-fc7349f4355b-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:23.126856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.126560    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84r8n\" (UniqueName: \"kubernetes.io/projected/8b541712-a45c-41a1-bfcc-fc7349f4355b-kube-api-access-84r8n\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:23.327737 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.324886    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"]
Oct 30 19:48:23.327737 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.325133    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h" podUID="edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" containerName="istio-proxy" containerID="cri-o://eb0dcad21c73806d6e9f49c305fc9b367e3d2d4ef2afea5e3f908e24c57d0fd2" gracePeriod=30
Oct 30 19:48:23.355913 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.355875    4524 generic.go:356] "Generic (PLEG): container finished" podID="8b541712-a45c-41a1-bfcc-fc7349f4355b" containerID="8f26347c52826768460bb3361f4ae45f59cbf74d26018b4db32e485883331751" exitCode=0
Oct 30 19:48:23.356103 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.355927    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"
Oct 30 19:48:23.356103 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.355954    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2" event={"ID":"8b541712-a45c-41a1-bfcc-fc7349f4355b","Type":"ContainerDied","Data":"8f26347c52826768460bb3361f4ae45f59cbf74d26018b4db32e485883331751"}
Oct 30 19:48:23.356103 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.356033    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2" event={"ID":"8b541712-a45c-41a1-bfcc-fc7349f4355b","Type":"ContainerDied","Data":"96af1adc5d91610f226782d19db3c7882ccba561ec2f133d91b9cbef28c20e56"}
Oct 30 19:48:23.356103 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.356066    4524 scope.go:122] "RemoveContainer" containerID="8f26347c52826768460bb3361f4ae45f59cbf74d26018b4db32e485883331751"
Oct 30 19:48:23.390425 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.390351    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"]
Oct 30 19:48:23.396354 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.396313    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-be6b74-d99d4c8d-k8tl2"]
Oct 30 19:48:23.440527 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.435505    4524 scope.go:122] "RemoveContainer" containerID="8f26347c52826768460bb3361f4ae45f59cbf74d26018b4db32e485883331751"
Oct 30 19:48:23.440527 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:48:23.436025    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f26347c52826768460bb3361f4ae45f59cbf74d26018b4db32e485883331751\": container with ID starting with 8f26347c52826768460bb3361f4ae45f59cbf74d26018b4db32e485883331751 not found: ID does not exist" containerID="8f26347c52826768460bb3361f4ae45f59cbf74d26018b4db32e485883331751"
Oct 30 19:48:23.440527 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:23.436062    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f26347c52826768460bb3361f4ae45f59cbf74d26018b4db32e485883331751"} err="failed to get container status \"8f26347c52826768460bb3361f4ae45f59cbf74d26018b4db32e485883331751\": rpc error: code = NotFound desc = could not find container \"8f26347c52826768460bb3361f4ae45f59cbf74d26018b4db32e485883331751\": container with ID starting with 8f26347c52826768460bb3361f4ae45f59cbf74d26018b4db32e485883331751 not found: ID does not exist"
Oct 30 19:48:24.391320 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:24.391286    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b541712-a45c-41a1-bfcc-fc7349f4355b" path="/var/lib/kubelet/pods/8b541712-a45c-41a1-bfcc-fc7349f4355b/volumes"
Oct 30 19:48:27.658909 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:27.658867    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz" podUID="e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0" containerName="main" containerID="cri-o://37cece85ba5947a81c70af51796893db4d7815e9c120f3d1303c8c570be603b7" gracePeriod=45
Oct 30 19:48:27.966527 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:27.966499    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:48:28.119009 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.118971    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-model-cache\") pod \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") "
Oct 30 19:48:28.119237 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.119042    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-tmp-dir\") pod \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") "
Oct 30 19:48:28.119325 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.119309    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-dshm\") pod \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") "
Oct 30 19:48:28.119392 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.119368    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-tls-certs\" (UniqueName: \"kubernetes.io/secret/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-tls-certs\") pod \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") "
Oct 30 19:48:28.119450 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.119397    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-kube-api-access-7nxm6\" (UniqueName: \"kubernetes.io/projected/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-kube-api-access-7nxm6\") pod \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") "
Oct 30 19:48:28.119450 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.119426    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-home\" (UniqueName: \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-home\") pod \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\" (UID: \"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0\") "
Oct 30 19:48:28.120529 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.120283    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-model-cache" pod "e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0" (UID: "e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:28.120529 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.120312    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-home" pod "e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0" (UID: "e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:28.120893 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.120847    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-tmp-dir" pod "e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0" (UID: "e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:28.122910 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.122883    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-tls-certs" pod "e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0" (UID: "e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:48:28.123449 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.123420    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-dshm" pod "e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0" (UID: "e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:28.123560 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.123531    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-kube-api-access-7nxm6" pod "e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0" (UID: "e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0"). InnerVolumeSpecName "kube-api-access-7nxm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:28.220919 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.220834    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nxm6\" (UniqueName: \"kubernetes.io/projected/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-kube-api-access-7nxm6\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:28.220919 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.220863    4524 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-home\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:28.220919 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.220876    4524 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-model-cache\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:28.220919 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.220889    4524 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-tmp-dir\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:28.220919 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.220901    4524 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-dshm\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:28.220919 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.220912    4524 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0-tls-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:28.447823 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.443816    4524 generic.go:356] "Generic (PLEG): container finished" podID="e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0" containerID="37cece85ba5947a81c70af51796893db4d7815e9c120f3d1303c8c570be603b7" exitCode=0
Oct 30 19:48:28.447823 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.443905    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"
Oct 30 19:48:28.447823 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.443906    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz" event={"ID":"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0","Type":"ContainerDied","Data":"37cece85ba5947a81c70af51796893db4d7815e9c120f3d1303c8c570be603b7"}
Oct 30 19:48:28.447823 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.444760    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz" event={"ID":"e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0","Type":"ContainerDied","Data":"eeb1b5c9a61584ae497225928909778a8a75a763dfbedc71559e869cd719f6fc"}
Oct 30 19:48:28.447823 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.444803    4524 scope.go:122] "RemoveContainer" containerID="37cece85ba5947a81c70af51796893db4d7815e9c120f3d1303c8c570be603b7"
Oct 30 19:48:28.459016 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.458992    4524 scope.go:122] "RemoveContainer" containerID="37cece85ba5947a81c70af51796893db4d7815e9c120f3d1303c8c570be603b7"
Oct 30 19:48:28.459691 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:48:28.459640    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37cece85ba5947a81c70af51796893db4d7815e9c120f3d1303c8c570be603b7\": container with ID starting with 37cece85ba5947a81c70af51796893db4d7815e9c120f3d1303c8c570be603b7 not found: ID does not exist" containerID="37cece85ba5947a81c70af51796893db4d7815e9c120f3d1303c8c570be603b7"
Oct 30 19:48:28.459806 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.459699    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37cece85ba5947a81c70af51796893db4d7815e9c120f3d1303c8c570be603b7"} err="failed to get container status \"37cece85ba5947a81c70af51796893db4d7815e9c120f3d1303c8c570be603b7\": rpc error: code = NotFound desc = could not find container \"37cece85ba5947a81c70af51796893db4d7815e9c120f3d1303c8c570be603b7\": container with ID starting with 37cece85ba5947a81c70af51796893db4d7815e9c120f3d1303c8c570be603b7 not found: ID does not exist"
Oct 30 19:48:28.475495 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.475432    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"]
Oct 30 19:48:28.483246 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.483211    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-2d5d53/distinct-w5-2d5d53-kserve-844959f975-gzsdz"]
Oct 30 19:48:28.620375 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.620348    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:48:28.625749 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.625727    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-kube-api-access-g4m2m\" (UniqueName: \"kubernetes.io/projected/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-kube-api-access-g4m2m\") pod \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") "
Oct 30 19:48:28.625884 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.625764    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istiod-ca-cert\") pod \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") "
Oct 30 19:48:28.625979 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.625893    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-credential-socket\") pod \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") "
Oct 30 19:48:28.625979 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.625952    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-data\") pod \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") "
Oct 30 19:48:28.626087 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.625985    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-workload-socket\") pod \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") "
Oct 30 19:48:28.626087 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.626061    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istiod-ca-cert" pod "edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" (UID: "edbb4bd8-4e56-4119-9170-f2e76cdb1eb2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:28.626202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.626142    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-credential-socket" pod "edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" (UID: "edbb4bd8-4e56-4119-9170-f2e76cdb1eb2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:28.626202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.626194    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-podinfo\") pod \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") "
Oct 30 19:48:28.626323 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.626231    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-data" pod "edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" (UID: "edbb4bd8-4e56-4119-9170-f2e76cdb1eb2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:28.626323 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.626242    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-workload-certs\") pod \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") "
Oct 30 19:48:28.626323 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.626288    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-envoy\") pod \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") "
Oct 30 19:48:28.626323 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.626313    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-workload-socket" pod "edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" (UID: "edbb4bd8-4e56-4119-9170-f2e76cdb1eb2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:28.626515 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.626323    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-token\" (UniqueName: \"kubernetes.io/projected/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-token\") pod \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\" (UID: \"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2\") "
Oct 30 19:48:28.626515 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.626466    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-workload-certs" pod "edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" (UID: "edbb4bd8-4e56-4119-9170-f2e76cdb1eb2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:28.626874 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.626850    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:28.627002 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.626877    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:28.627002 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.626892    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:28.627002 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.626905    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:28.627002 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.626919    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:28.628346 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.628320    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-kube-api-access-g4m2m" pod "edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" (UID: "edbb4bd8-4e56-4119-9170-f2e76cdb1eb2"). InnerVolumeSpecName "kube-api-access-g4m2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:28.628520 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.628487    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-podinfo" pod "edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" (UID: "edbb4bd8-4e56-4119-9170-f2e76cdb1eb2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:48:28.628520 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.628490    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-envoy" pod "edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" (UID: "edbb4bd8-4e56-4119-9170-f2e76cdb1eb2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:28.628654 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.628545    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-token" pod "edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" (UID: "edbb4bd8-4e56-4119-9170-f2e76cdb1eb2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:28.727603 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.727536    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:28.727603 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.727563    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:28.727603 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.727575    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:28.727603 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:28.727584    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4m2m\" (UniqueName: \"kubernetes.io/projected/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2-kube-api-access-g4m2m\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:29.452833 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:29.452800    4524 generic.go:356] "Generic (PLEG): container finished" podID="edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" containerID="eb0dcad21c73806d6e9f49c305fc9b367e3d2d4ef2afea5e3f908e24c57d0fd2" exitCode=0
Oct 30 19:48:29.453071 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:29.452842    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"
Oct 30 19:48:29.453071 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:29.452866    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h" event={"ID":"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2","Type":"ContainerDied","Data":"eb0dcad21c73806d6e9f49c305fc9b367e3d2d4ef2afea5e3f908e24c57d0fd2"}
Oct 30 19:48:29.453071 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:29.452932    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h" event={"ID":"edbb4bd8-4e56-4119-9170-f2e76cdb1eb2","Type":"ContainerDied","Data":"104c4322c0370f4cf3da116945ecb5a759889e891bb893f07f736252e0710663"}
Oct 30 19:48:29.453071 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:29.452962    4524 scope.go:122] "RemoveContainer" containerID="eb0dcad21c73806d6e9f49c305fc9b367e3d2d4ef2afea5e3f908e24c57d0fd2"
Oct 30 19:48:29.463384 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:29.463369    4524 scope.go:122] "RemoveContainer" containerID="eb0dcad21c73806d6e9f49c305fc9b367e3d2d4ef2afea5e3f908e24c57d0fd2"
Oct 30 19:48:29.463615 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:48:29.463597    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb0dcad21c73806d6e9f49c305fc9b367e3d2d4ef2afea5e3f908e24c57d0fd2\": container with ID starting with eb0dcad21c73806d6e9f49c305fc9b367e3d2d4ef2afea5e3f908e24c57d0fd2 not found: ID does not exist" containerID="eb0dcad21c73806d6e9f49c305fc9b367e3d2d4ef2afea5e3f908e24c57d0fd2"
Oct 30 19:48:29.463664 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:29.463623    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb0dcad21c73806d6e9f49c305fc9b367e3d2d4ef2afea5e3f908e24c57d0fd2"} err="failed to get container status \"eb0dcad21c73806d6e9f49c305fc9b367e3d2d4ef2afea5e3f908e24c57d0fd2\": rpc error: code = NotFound desc = could not find container \"eb0dcad21c73806d6e9f49c305fc9b367e3d2d4ef2afea5e3f908e24c57d0fd2\": container with ID starting with eb0dcad21c73806d6e9f49c305fc9b367e3d2d4ef2afea5e3f908e24c57d0fd2 not found: ID does not exist"
Oct 30 19:48:29.477737 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:29.477713    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"]
Oct 30 19:48:29.490655 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:29.490631    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b281d5-openshift-default-fcfb8bfc5-xd88h"]
Oct 30 19:48:30.392011 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:30.391976    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0" path="/var/lib/kubelet/pods/e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0/volumes"
Oct 30 19:48:30.392677 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:30.392655    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" path="/var/lib/kubelet/pods/edbb4bd8-4e56-4119-9170-f2e76cdb1eb2/volumes"
Oct 30 19:48:32.504587 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:32.504537    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb" podUID="1affe672-c981-43ad-bd2c-099648e4be4c" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:48:32.505077 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:32.504657    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq" podUID="7ac2f9eb-a945-483a-9c0a-7ad91b8fd618" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:48:32.505077 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:32.504707    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh" podUID="3689eb31-fbba-4cbf-a355-7056ccec100b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:48:32.505077 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:32.504969    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h" podUID="db66fd6a-afe9-47e7-815c-5c66c86081d8" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:48:34.247973 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.247936    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"]
Oct 30 19:48:34.248474 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.248194    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2" podUID="a03edaaa-eafc-4c25-846a-ff9fab080afc" containerName="maas-api" containerID="cri-o://10fe9cce3096067886f9ef3cedda546a783e986b099571720a0a862d2e05e990" gracePeriod=30
Oct 30 19:48:34.355968 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.355925    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"]
Oct 30 19:48:34.356202 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.356151    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g" podUID="29e3ef22-7657-4fe5-803e-75a93978aced" containerName="maas-api" containerID="cri-o://f4909b92b9fe474a0c65ad8d38fde96588fd10cf02c34c21aeb7ffea2b43afd5" gracePeriod=30
Oct 30 19:48:34.436759 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.436716    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g" podUID="29e3ef22-7657-4fe5-803e-75a93978aced" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.70:8443/health\": dial tcp 10.133.0.70:8443: connect: connection refused"
Oct 30 19:48:34.522275 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.518580    4524 generic.go:356] "Generic (PLEG): container finished" podID="29e3ef22-7657-4fe5-803e-75a93978aced" containerID="f4909b92b9fe474a0c65ad8d38fde96588fd10cf02c34c21aeb7ffea2b43afd5" exitCode=0
Oct 30 19:48:34.522275 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.518802    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g" event={"ID":"29e3ef22-7657-4fe5-803e-75a93978aced","Type":"ContainerDied","Data":"f4909b92b9fe474a0c65ad8d38fde96588fd10cf02c34c21aeb7ffea2b43afd5"}
Oct 30 19:48:34.522275 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.522090    4524 generic.go:356] "Generic (PLEG): container finished" podID="a03edaaa-eafc-4c25-846a-ff9fab080afc" containerID="10fe9cce3096067886f9ef3cedda546a783e986b099571720a0a862d2e05e990" exitCode=0
Oct 30 19:48:34.522275 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.522232    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2" event={"ID":"a03edaaa-eafc-4c25-846a-ff9fab080afc","Type":"ContainerDied","Data":"10fe9cce3096067886f9ef3cedda546a783e986b099571720a0a862d2e05e990"}
Oct 30 19:48:34.680042 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.679940    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"
Oct 30 19:48:34.717811 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.716563    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a03edaaa-eafc-4c25-846a-ff9fab080afc-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a03edaaa-eafc-4c25-846a-ff9fab080afc-metrics-certs\") pod \"a03edaaa-eafc-4c25-846a-ff9fab080afc\" (UID: \"a03edaaa-eafc-4c25-846a-ff9fab080afc\") "
Oct 30 19:48:34.717811 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.716614    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a03edaaa-eafc-4c25-846a-ff9fab080afc-kube-api-access-r7vns\" (UniqueName: \"kubernetes.io/projected/a03edaaa-eafc-4c25-846a-ff9fab080afc-kube-api-access-r7vns\") pod \"a03edaaa-eafc-4c25-846a-ff9fab080afc\" (UID: \"a03edaaa-eafc-4c25-846a-ff9fab080afc\") "
Oct 30 19:48:34.717811 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.716665    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a03edaaa-eafc-4c25-846a-ff9fab080afc-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a03edaaa-eafc-4c25-846a-ff9fab080afc-openshift-service-ca\") pod \"a03edaaa-eafc-4c25-846a-ff9fab080afc\" (UID: \"a03edaaa-eafc-4c25-846a-ff9fab080afc\") "
Oct 30 19:48:34.717811 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.716786    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a03edaaa-eafc-4c25-846a-ff9fab080afc-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a03edaaa-eafc-4c25-846a-ff9fab080afc-maas-api-tls\") pod \"a03edaaa-eafc-4c25-846a-ff9fab080afc\" (UID: \"a03edaaa-eafc-4c25-846a-ff9fab080afc\") "
Oct 30 19:48:34.718941 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.718902    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a03edaaa-eafc-4c25-846a-ff9fab080afc-openshift-service-ca" pod "a03edaaa-eafc-4c25-846a-ff9fab080afc" (UID: "a03edaaa-eafc-4c25-846a-ff9fab080afc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:34.720071 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.720039    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a03edaaa-eafc-4c25-846a-ff9fab080afc-kube-api-access-r7vns" pod "a03edaaa-eafc-4c25-846a-ff9fab080afc" (UID: "a03edaaa-eafc-4c25-846a-ff9fab080afc"). InnerVolumeSpecName "kube-api-access-r7vns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:34.721779 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.721744    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a03edaaa-eafc-4c25-846a-ff9fab080afc-metrics-certs" pod "a03edaaa-eafc-4c25-846a-ff9fab080afc" (UID: "a03edaaa-eafc-4c25-846a-ff9fab080afc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:48:34.722056 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.722025    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a03edaaa-eafc-4c25-846a-ff9fab080afc-maas-api-tls" pod "a03edaaa-eafc-4c25-846a-ff9fab080afc" (UID: "a03edaaa-eafc-4c25-846a-ff9fab080afc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:48:34.770019 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.769994    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"
Oct 30 19:48:34.817824 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.817791    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29e3ef22-7657-4fe5-803e-75a93978aced-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29e3ef22-7657-4fe5-803e-75a93978aced-openshift-service-ca\") pod \"29e3ef22-7657-4fe5-803e-75a93978aced\" (UID: \"29e3ef22-7657-4fe5-803e-75a93978aced\") "
Oct 30 19:48:34.817995 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.817961    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/29e3ef22-7657-4fe5-803e-75a93978aced-metrics-certs\" (UniqueName: \"kubernetes.io/secret/29e3ef22-7657-4fe5-803e-75a93978aced-metrics-certs\") pod \"29e3ef22-7657-4fe5-803e-75a93978aced\" (UID: \"29e3ef22-7657-4fe5-803e-75a93978aced\") "
Oct 30 19:48:34.818052 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.818032    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29e3ef22-7657-4fe5-803e-75a93978aced-kube-api-access-wn2tp\" (UniqueName: \"kubernetes.io/projected/29e3ef22-7657-4fe5-803e-75a93978aced-kube-api-access-wn2tp\") pod \"29e3ef22-7657-4fe5-803e-75a93978aced\" (UID: \"29e3ef22-7657-4fe5-803e-75a93978aced\") "
Oct 30 19:48:34.818103 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.818087    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/29e3ef22-7657-4fe5-803e-75a93978aced-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29e3ef22-7657-4fe5-803e-75a93978aced-maas-api-tls\") pod \"29e3ef22-7657-4fe5-803e-75a93978aced\" (UID: \"29e3ef22-7657-4fe5-803e-75a93978aced\") "
Oct 30 19:48:34.818277 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.818250    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29e3ef22-7657-4fe5-803e-75a93978aced-openshift-service-ca" pod "29e3ef22-7657-4fe5-803e-75a93978aced" (UID: "29e3ef22-7657-4fe5-803e-75a93978aced"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:34.818590 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.818491    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a03edaaa-eafc-4c25-846a-ff9fab080afc-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:34.818590 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.818513    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7vns\" (UniqueName: \"kubernetes.io/projected/a03edaaa-eafc-4c25-846a-ff9fab080afc-kube-api-access-r7vns\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:34.818590 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.818529    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a03edaaa-eafc-4c25-846a-ff9fab080afc-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:34.818590 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.818543    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a03edaaa-eafc-4c25-846a-ff9fab080afc-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:34.818590 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.818556    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29e3ef22-7657-4fe5-803e-75a93978aced-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:34.820157 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.820129    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29e3ef22-7657-4fe5-803e-75a93978aced-metrics-certs" pod "29e3ef22-7657-4fe5-803e-75a93978aced" (UID: "29e3ef22-7657-4fe5-803e-75a93978aced"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:48:34.820924 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.820898    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29e3ef22-7657-4fe5-803e-75a93978aced-maas-api-tls" pod "29e3ef22-7657-4fe5-803e-75a93978aced" (UID: "29e3ef22-7657-4fe5-803e-75a93978aced"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:48:34.820996 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.820908    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29e3ef22-7657-4fe5-803e-75a93978aced-kube-api-access-wn2tp" pod "29e3ef22-7657-4fe5-803e-75a93978aced" (UID: "29e3ef22-7657-4fe5-803e-75a93978aced"). InnerVolumeSpecName "kube-api-access-wn2tp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:34.919776 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.919733    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wn2tp\" (UniqueName: \"kubernetes.io/projected/29e3ef22-7657-4fe5-803e-75a93978aced-kube-api-access-wn2tp\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:34.919776 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.919772    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29e3ef22-7657-4fe5-803e-75a93978aced-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:34.919776 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:34.919781    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29e3ef22-7657-4fe5-803e-75a93978aced-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:35.533400 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:35.533312    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"
Oct 30 19:48:35.533868 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:35.533312    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2" event={"ID":"a03edaaa-eafc-4c25-846a-ff9fab080afc","Type":"ContainerDied","Data":"1a30c391c2d46f57a4e2b48f8eabc22634d3dd4fe97c57be284e7a60b9f4615c"}
Oct 30 19:48:35.533868 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:35.533482    4524 scope.go:122] "RemoveContainer" containerID="10fe9cce3096067886f9ef3cedda546a783e986b099571720a0a862d2e05e990"
Oct 30 19:48:35.535699 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:35.535675    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g" event={"ID":"29e3ef22-7657-4fe5-803e-75a93978aced","Type":"ContainerDied","Data":"cc866729d730621d51fe16ae3a237cb83861285ba6f21a683053d34da52a9878"}
Oct 30 19:48:35.535879 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:35.535797    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"
Oct 30 19:48:35.546350 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:35.546328    4524 scope.go:122] "RemoveContainer" containerID="f4909b92b9fe474a0c65ad8d38fde96588fd10cf02c34c21aeb7ffea2b43afd5"
Oct 30 19:48:35.564996 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:35.564968    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"]
Oct 30 19:48:35.569712 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:35.569689    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-2d5d53-7867cfcfdb-hlj8g"]
Oct 30 19:48:35.582778 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:35.582750    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"]
Oct 30 19:48:35.587447 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:35.587422    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-27b5d5-5f5bdf6b5c-xhbf2"]
Oct 30 19:48:36.392260 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:36.392224    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29e3ef22-7657-4fe5-803e-75a93978aced" path="/var/lib/kubelet/pods/29e3ef22-7657-4fe5-803e-75a93978aced/volumes"
Oct 30 19:48:36.392708 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:36.392691    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a03edaaa-eafc-4c25-846a-ff9fab080afc" path="/var/lib/kubelet/pods/a03edaaa-eafc-4c25-846a-ff9fab080afc/volumes"
Oct 30 19:48:37.981490 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:37.981451    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"]
Oct 30 19:48:37.981986 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:37.981693    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv" podUID="558458ff-e6e8-41a9-9b1b-c9d0dc6c708e" containerName="maas-api" containerID="cri-o://f55435c4db2701d21a2cc382511a52433e80a0e5e5f5feccc999babd1938cf21" gracePeriod=30
Oct 30 19:48:38.349646 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.349616    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"
Oct 30 19:48:38.469766 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.469734    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-openshift-service-ca\") pod \"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e\" (UID: \"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e\") "
Oct 30 19:48:38.469943 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.469882    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-maas-api-tls\") pod \"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e\" (UID: \"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e\") "
Oct 30 19:48:38.469943 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.469918    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-metrics-certs\") pod \"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e\" (UID: \"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e\") "
Oct 30 19:48:38.470069 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.469957    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-kube-api-access-2jf97\" (UniqueName: \"kubernetes.io/projected/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-kube-api-access-2jf97\") pod \"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e\" (UID: \"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e\") "
Oct 30 19:48:38.470166 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.470138    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-openshift-service-ca" pod "558458ff-e6e8-41a9-9b1b-c9d0dc6c708e" (UID: "558458ff-e6e8-41a9-9b1b-c9d0dc6c708e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:38.470542 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.470516    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:38.472831 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.472764    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-maas-api-tls" pod "558458ff-e6e8-41a9-9b1b-c9d0dc6c708e" (UID: "558458ff-e6e8-41a9-9b1b-c9d0dc6c708e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:48:38.472943 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.472829    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-kube-api-access-2jf97" pod "558458ff-e6e8-41a9-9b1b-c9d0dc6c708e" (UID: "558458ff-e6e8-41a9-9b1b-c9d0dc6c708e"). InnerVolumeSpecName "kube-api-access-2jf97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:38.472943 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.472887    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-metrics-certs" pod "558458ff-e6e8-41a9-9b1b-c9d0dc6c708e" (UID: "558458ff-e6e8-41a9-9b1b-c9d0dc6c708e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:48:38.560629 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.560546    4524 generic.go:356] "Generic (PLEG): container finished" podID="558458ff-e6e8-41a9-9b1b-c9d0dc6c708e" containerID="f55435c4db2701d21a2cc382511a52433e80a0e5e5f5feccc999babd1938cf21" exitCode=0
Oct 30 19:48:38.560629 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.560597    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"
Oct 30 19:48:38.560894 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.560627    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv" event={"ID":"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e","Type":"ContainerDied","Data":"f55435c4db2701d21a2cc382511a52433e80a0e5e5f5feccc999babd1938cf21"}
Oct 30 19:48:38.560894 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.560683    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv" event={"ID":"558458ff-e6e8-41a9-9b1b-c9d0dc6c708e","Type":"ContainerDied","Data":"bc3eebe90a198f0302c738a880fd5f134b4c91aef2eedeba8e8aa1ee35733b51"}
Oct 30 19:48:38.560894 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.560710    4524 scope.go:122] "RemoveContainer" containerID="f55435c4db2701d21a2cc382511a52433e80a0e5e5f5feccc999babd1938cf21"
Oct 30 19:48:38.571131 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.571109    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:38.571248 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.571134    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:38.571248 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.571145    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jf97\" (UniqueName: \"kubernetes.io/projected/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e-kube-api-access-2jf97\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:38.572390 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.572372    4524 scope.go:122] "RemoveContainer" containerID="f55435c4db2701d21a2cc382511a52433e80a0e5e5f5feccc999babd1938cf21"
Oct 30 19:48:38.572748 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:48:38.572712    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f55435c4db2701d21a2cc382511a52433e80a0e5e5f5feccc999babd1938cf21\": container with ID starting with f55435c4db2701d21a2cc382511a52433e80a0e5e5f5feccc999babd1938cf21 not found: ID does not exist" containerID="f55435c4db2701d21a2cc382511a52433e80a0e5e5f5feccc999babd1938cf21"
Oct 30 19:48:38.572748 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.572743    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f55435c4db2701d21a2cc382511a52433e80a0e5e5f5feccc999babd1938cf21"} err="failed to get container status \"f55435c4db2701d21a2cc382511a52433e80a0e5e5f5feccc999babd1938cf21\": rpc error: code = NotFound desc = could not find container \"f55435c4db2701d21a2cc382511a52433e80a0e5e5f5feccc999babd1938cf21\": container with ID starting with f55435c4db2701d21a2cc382511a52433e80a0e5e5f5feccc999babd1938cf21 not found: ID does not exist"
Oct 30 19:48:38.590336 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.590315    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"]
Oct 30 19:48:38.608795 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:38.608775    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-8f3ff4-7b7969cb87-k7bwv"]
Oct 30 19:48:40.416632 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:40.416595    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="558458ff-e6e8-41a9-9b1b-c9d0dc6c708e" path="/var/lib/kubelet/pods/558458ff-e6e8-41a9-9b1b-c9d0dc6c708e/volumes"
Oct 30 19:48:42.006221 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:41.996950    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h"
Oct 30 19:48:42.060882 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.060858    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb"
Oct 30 19:48:42.122954 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.122924    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1affe672-c981-43ad-bd2c-099648e4be4c-kube-api-access-726kc\" (UniqueName: \"kubernetes.io/projected/1affe672-c981-43ad-bd2c-099648e4be4c-kube-api-access-726kc\") pod \"1affe672-c981-43ad-bd2c-099648e4be4c\" (UID: \"1affe672-c981-43ad-bd2c-099648e4be4c\") "
Oct 30 19:48:42.123120 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.123102    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1affe672-c981-43ad-bd2c-099648e4be4c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1affe672-c981-43ad-bd2c-099648e4be4c-plugins-config-volume\") pod \"1affe672-c981-43ad-bd2c-099648e4be4c\" (UID: \"1affe672-c981-43ad-bd2c-099648e4be4c\") "
Oct 30 19:48:42.123209 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.123149    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db66fd6a-afe9-47e7-815c-5c66c86081d8-kube-api-access-sjblp\" (UniqueName: \"kubernetes.io/projected/db66fd6a-afe9-47e7-815c-5c66c86081d8-kube-api-access-sjblp\") pod \"db66fd6a-afe9-47e7-815c-5c66c86081d8\" (UID: \"db66fd6a-afe9-47e7-815c-5c66c86081d8\") "
Oct 30 19:48:42.123271 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.123248    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db66fd6a-afe9-47e7-815c-5c66c86081d8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db66fd6a-afe9-47e7-815c-5c66c86081d8-plugins-config-volume\") pod \"db66fd6a-afe9-47e7-815c-5c66c86081d8\" (UID: \"db66fd6a-afe9-47e7-815c-5c66c86081d8\") "
Oct 30 19:48:42.123541 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.123508    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1affe672-c981-43ad-bd2c-099648e4be4c-plugins-config-volume" pod "1affe672-c981-43ad-bd2c-099648e4be4c" (UID: "1affe672-c981-43ad-bd2c-099648e4be4c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:42.123701 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.123678    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1affe672-c981-43ad-bd2c-099648e4be4c-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:42.123778 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.123708    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db66fd6a-afe9-47e7-815c-5c66c86081d8-plugins-config-volume" pod "db66fd6a-afe9-47e7-815c-5c66c86081d8" (UID: "db66fd6a-afe9-47e7-815c-5c66c86081d8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:42.125322 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.125300    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1affe672-c981-43ad-bd2c-099648e4be4c-kube-api-access-726kc" pod "1affe672-c981-43ad-bd2c-099648e4be4c" (UID: "1affe672-c981-43ad-bd2c-099648e4be4c"). InnerVolumeSpecName "kube-api-access-726kc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:42.125412 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.125323    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db66fd6a-afe9-47e7-815c-5c66c86081d8-kube-api-access-sjblp" pod "db66fd6a-afe9-47e7-815c-5c66c86081d8" (UID: "db66fd6a-afe9-47e7-815c-5c66c86081d8"). InnerVolumeSpecName "kube-api-access-sjblp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:42.224970 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.224883    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sjblp\" (UniqueName: \"kubernetes.io/projected/db66fd6a-afe9-47e7-815c-5c66c86081d8-kube-api-access-sjblp\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:42.224970 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.224921    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db66fd6a-afe9-47e7-815c-5c66c86081d8-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:42.224970 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.224937    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-726kc\" (UniqueName: \"kubernetes.io/projected/1affe672-c981-43ad-bd2c-099648e4be4c-kube-api-access-726kc\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:42.441053 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.441014    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"]
Oct 30 19:48:42.441293 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.441262    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs" podUID="6fb90c94-7e88-4cd8-96c1-f779013b1694" containerName="maas-api" containerID="cri-o://67f952639a0536feff2cb15669432494070cf290a293463f09a54fc9d5c5ed32" gracePeriod=30
Oct 30 19:48:42.510479 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.509827    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh" podUID="3689eb31-fbba-4cbf-a355-7056ccec100b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:48:42.511710 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.511651    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq" podUID="7ac2f9eb-a945-483a-9c0a-7ad91b8fd618" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:48:42.596617 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.596585    4524 generic.go:356] "Generic (PLEG): container finished" podID="db66fd6a-afe9-47e7-815c-5c66c86081d8" containerID="37569f1588ec4102c1cb7a3ece5b5ac88d7408ec11a574ef809f03d51a2ac79a" exitCode=0
Oct 30 19:48:42.596745 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.596662    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h" event={"ID":"db66fd6a-afe9-47e7-815c-5c66c86081d8","Type":"ContainerDied","Data":"37569f1588ec4102c1cb7a3ece5b5ac88d7408ec11a574ef809f03d51a2ac79a"}
Oct 30 19:48:42.596745 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.596724    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h" event={"ID":"db66fd6a-afe9-47e7-815c-5c66c86081d8","Type":"ContainerDied","Data":"e106724ce31b84d9c11f7cae7645b5c906658ec8b14a66f6a3735480684da9a0"}
Oct 30 19:48:42.596881 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.596755    4524 scope.go:122] "RemoveContainer" containerID="37569f1588ec4102c1cb7a3ece5b5ac88d7408ec11a574ef809f03d51a2ac79a"
Oct 30 19:48:42.596935 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.596886    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h"
Oct 30 19:48:42.598888 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.598861    4524 generic.go:356] "Generic (PLEG): container finished" podID="1affe672-c981-43ad-bd2c-099648e4be4c" containerID="594a7a82458e83016d71647fd6c49048dfdc29051ba1b6e9d86a73e417db102f" exitCode=0
Oct 30 19:48:42.600939 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.600917    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb" event={"ID":"1affe672-c981-43ad-bd2c-099648e4be4c","Type":"ContainerDied","Data":"594a7a82458e83016d71647fd6c49048dfdc29051ba1b6e9d86a73e417db102f"}
Oct 30 19:48:42.601054 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.600973    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb" event={"ID":"1affe672-c981-43ad-bd2c-099648e4be4c","Type":"ContainerDied","Data":"cd1238d2336274f0c9a9e3896b64cfacd905f3e8928720a6f190b62b2e3f1e01"}
Oct 30 19:48:42.601054 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.601028    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb"
Oct 30 19:48:42.603321 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.603226    4524 generic.go:356] "Generic (PLEG): container finished" podID="6fb90c94-7e88-4cd8-96c1-f779013b1694" containerID="67f952639a0536feff2cb15669432494070cf290a293463f09a54fc9d5c5ed32" exitCode=0
Oct 30 19:48:42.603321 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.603282    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs" event={"ID":"6fb90c94-7e88-4cd8-96c1-f779013b1694","Type":"ContainerDied","Data":"67f952639a0536feff2cb15669432494070cf290a293463f09a54fc9d5c5ed32"}
Oct 30 19:48:42.620263 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.620061    4524 scope.go:122] "RemoveContainer" containerID="37569f1588ec4102c1cb7a3ece5b5ac88d7408ec11a574ef809f03d51a2ac79a"
Oct 30 19:48:42.620680 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:48:42.620610    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37569f1588ec4102c1cb7a3ece5b5ac88d7408ec11a574ef809f03d51a2ac79a\": container with ID starting with 37569f1588ec4102c1cb7a3ece5b5ac88d7408ec11a574ef809f03d51a2ac79a not found: ID does not exist" containerID="37569f1588ec4102c1cb7a3ece5b5ac88d7408ec11a574ef809f03d51a2ac79a"
Oct 30 19:48:42.620783 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.620687    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37569f1588ec4102c1cb7a3ece5b5ac88d7408ec11a574ef809f03d51a2ac79a"} err="failed to get container status \"37569f1588ec4102c1cb7a3ece5b5ac88d7408ec11a574ef809f03d51a2ac79a\": rpc error: code = NotFound desc = could not find container \"37569f1588ec4102c1cb7a3ece5b5ac88d7408ec11a574ef809f03d51a2ac79a\": container with ID starting with 37569f1588ec4102c1cb7a3ece5b5ac88d7408ec11a574ef809f03d51a2ac79a not found: ID does not exist"
Oct 30 19:48:42.620783 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.620728    4524 scope.go:122] "RemoveContainer" containerID="594a7a82458e83016d71647fd6c49048dfdc29051ba1b6e9d86a73e417db102f"
Oct 30 19:48:42.632195 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.632162    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h"]
Oct 30 19:48:42.632728 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.632711    4524 scope.go:122] "RemoveContainer" containerID="594a7a82458e83016d71647fd6c49048dfdc29051ba1b6e9d86a73e417db102f"
Oct 30 19:48:42.633048 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:48:42.633027    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"594a7a82458e83016d71647fd6c49048dfdc29051ba1b6e9d86a73e417db102f\": container with ID starting with 594a7a82458e83016d71647fd6c49048dfdc29051ba1b6e9d86a73e417db102f not found: ID does not exist" containerID="594a7a82458e83016d71647fd6c49048dfdc29051ba1b6e9d86a73e417db102f"
Oct 30 19:48:42.633129 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.633055    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"594a7a82458e83016d71647fd6c49048dfdc29051ba1b6e9d86a73e417db102f"} err="failed to get container status \"594a7a82458e83016d71647fd6c49048dfdc29051ba1b6e9d86a73e417db102f\": rpc error: code = NotFound desc = could not find container \"594a7a82458e83016d71647fd6c49048dfdc29051ba1b6e9d86a73e417db102f\": container with ID starting with 594a7a82458e83016d71647fd6c49048dfdc29051ba1b6e9d86a73e417db102f not found: ID does not exist"
Oct 30 19:48:42.641893 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.641869    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-27b5d5-7bffddc884-qdb9h"]
Oct 30 19:48:42.653332 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.653312    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb"]
Oct 30 19:48:42.664184 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.664149    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-27b5d5-8644c44ff-466fb"]
Oct 30 19:48:42.807259 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.805842    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"
Oct 30 19:48:42.937244 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.935926    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6fb90c94-7e88-4cd8-96c1-f779013b1694-kube-api-access-j9xf2\" (UniqueName: \"kubernetes.io/projected/6fb90c94-7e88-4cd8-96c1-f779013b1694-kube-api-access-j9xf2\") pod \"6fb90c94-7e88-4cd8-96c1-f779013b1694\" (UID: \"6fb90c94-7e88-4cd8-96c1-f779013b1694\") "
Oct 30 19:48:42.937244 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.936023    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6fb90c94-7e88-4cd8-96c1-f779013b1694-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fb90c94-7e88-4cd8-96c1-f779013b1694-metrics-certs\") pod \"6fb90c94-7e88-4cd8-96c1-f779013b1694\" (UID: \"6fb90c94-7e88-4cd8-96c1-f779013b1694\") "
Oct 30 19:48:42.937244 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.936100    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6fb90c94-7e88-4cd8-96c1-f779013b1694-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb90c94-7e88-4cd8-96c1-f779013b1694-maas-api-tls\") pod \"6fb90c94-7e88-4cd8-96c1-f779013b1694\" (UID: \"6fb90c94-7e88-4cd8-96c1-f779013b1694\") "
Oct 30 19:48:42.937244 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.936160    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6fb90c94-7e88-4cd8-96c1-f779013b1694-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb90c94-7e88-4cd8-96c1-f779013b1694-openshift-service-ca\") pod \"6fb90c94-7e88-4cd8-96c1-f779013b1694\" (UID: \"6fb90c94-7e88-4cd8-96c1-f779013b1694\") "
Oct 30 19:48:42.937600 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.937280    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fb90c94-7e88-4cd8-96c1-f779013b1694-openshift-service-ca" pod "6fb90c94-7e88-4cd8-96c1-f779013b1694" (UID: "6fb90c94-7e88-4cd8-96c1-f779013b1694"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:42.957018 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.952486    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fb90c94-7e88-4cd8-96c1-f779013b1694-maas-api-tls" pod "6fb90c94-7e88-4cd8-96c1-f779013b1694" (UID: "6fb90c94-7e88-4cd8-96c1-f779013b1694"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:48:42.960449 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.957551    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fb90c94-7e88-4cd8-96c1-f779013b1694-kube-api-access-j9xf2" pod "6fb90c94-7e88-4cd8-96c1-f779013b1694" (UID: "6fb90c94-7e88-4cd8-96c1-f779013b1694"). InnerVolumeSpecName "kube-api-access-j9xf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:42.960449 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:42.958026    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fb90c94-7e88-4cd8-96c1-f779013b1694-metrics-certs" pod "6fb90c94-7e88-4cd8-96c1-f779013b1694" (UID: "6fb90c94-7e88-4cd8-96c1-f779013b1694"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:48:43.037602 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.037576    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb90c94-7e88-4cd8-96c1-f779013b1694-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:43.037602 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.037604    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb90c94-7e88-4cd8-96c1-f779013b1694-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:43.037928 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.037614    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9xf2\" (UniqueName: \"kubernetes.io/projected/6fb90c94-7e88-4cd8-96c1-f779013b1694-kube-api-access-j9xf2\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:43.037928 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.037624    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fb90c94-7e88-4cd8-96c1-f779013b1694-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:43.218191 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.217672    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq"
Oct 30 19:48:43.275414 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.275385    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh"
Oct 30 19:48:43.340905 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.340876    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ac2f9eb-a945-483a-9c0a-7ad91b8fd618-kube-api-access-rmhfj\" (UniqueName: \"kubernetes.io/projected/7ac2f9eb-a945-483a-9c0a-7ad91b8fd618-kube-api-access-rmhfj\") pod \"7ac2f9eb-a945-483a-9c0a-7ad91b8fd618\" (UID: \"7ac2f9eb-a945-483a-9c0a-7ad91b8fd618\") "
Oct 30 19:48:43.341058 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.340911    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ac2f9eb-a945-483a-9c0a-7ad91b8fd618-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ac2f9eb-a945-483a-9c0a-7ad91b8fd618-plugins-config-volume\") pod \"7ac2f9eb-a945-483a-9c0a-7ad91b8fd618\" (UID: \"7ac2f9eb-a945-483a-9c0a-7ad91b8fd618\") "
Oct 30 19:48:43.341058 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.340937    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3689eb31-fbba-4cbf-a355-7056ccec100b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3689eb31-fbba-4cbf-a355-7056ccec100b-plugins-config-volume\") pod \"3689eb31-fbba-4cbf-a355-7056ccec100b\" (UID: \"3689eb31-fbba-4cbf-a355-7056ccec100b\") "
Oct 30 19:48:43.341058 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.340982    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3689eb31-fbba-4cbf-a355-7056ccec100b-kube-api-access-9bw7d\" (UniqueName: \"kubernetes.io/projected/3689eb31-fbba-4cbf-a355-7056ccec100b-kube-api-access-9bw7d\") pod \"3689eb31-fbba-4cbf-a355-7056ccec100b\" (UID: \"3689eb31-fbba-4cbf-a355-7056ccec100b\") "
Oct 30 19:48:43.341327 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.341297    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ac2f9eb-a945-483a-9c0a-7ad91b8fd618-plugins-config-volume" pod "7ac2f9eb-a945-483a-9c0a-7ad91b8fd618" (UID: "7ac2f9eb-a945-483a-9c0a-7ad91b8fd618"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:43.341460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.341418    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ac2f9eb-a945-483a-9c0a-7ad91b8fd618-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:43.341460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.341417    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3689eb31-fbba-4cbf-a355-7056ccec100b-plugins-config-volume" pod "3689eb31-fbba-4cbf-a355-7056ccec100b" (UID: "3689eb31-fbba-4cbf-a355-7056ccec100b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:43.343067 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.343050    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ac2f9eb-a945-483a-9c0a-7ad91b8fd618-kube-api-access-rmhfj" pod "7ac2f9eb-a945-483a-9c0a-7ad91b8fd618" (UID: "7ac2f9eb-a945-483a-9c0a-7ad91b8fd618"). InnerVolumeSpecName "kube-api-access-rmhfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:43.343128 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.343109    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3689eb31-fbba-4cbf-a355-7056ccec100b-kube-api-access-9bw7d" pod "3689eb31-fbba-4cbf-a355-7056ccec100b" (UID: "3689eb31-fbba-4cbf-a355-7056ccec100b"). InnerVolumeSpecName "kube-api-access-9bw7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:43.443025 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.442993    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmhfj\" (UniqueName: \"kubernetes.io/projected/7ac2f9eb-a945-483a-9c0a-7ad91b8fd618-kube-api-access-rmhfj\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:43.443025 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.443017    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3689eb31-fbba-4cbf-a355-7056ccec100b-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:43.443025 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.443027    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bw7d\" (UniqueName: \"kubernetes.io/projected/3689eb31-fbba-4cbf-a355-7056ccec100b-kube-api-access-9bw7d\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:43.613035 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.612945    4524 generic.go:356] "Generic (PLEG): container finished" podID="7ac2f9eb-a945-483a-9c0a-7ad91b8fd618" containerID="e866f74e712d90cf48a0f818e0cdfdcbee70f49fab1722a1963d019b54907a8c" exitCode=0
Oct 30 19:48:43.613035 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.612995    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq"
Oct 30 19:48:43.613292 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.613033    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq" event={"ID":"7ac2f9eb-a945-483a-9c0a-7ad91b8fd618","Type":"ContainerDied","Data":"e866f74e712d90cf48a0f818e0cdfdcbee70f49fab1722a1963d019b54907a8c"}
Oct 30 19:48:43.613292 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.613097    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq" event={"ID":"7ac2f9eb-a945-483a-9c0a-7ad91b8fd618","Type":"ContainerDied","Data":"675a04c8942050541aa10d69161a29aef1ae8d8ab722d796fa33c134059f18a3"}
Oct 30 19:48:43.613292 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.613128    4524 scope.go:122] "RemoveContainer" containerID="e866f74e712d90cf48a0f818e0cdfdcbee70f49fab1722a1963d019b54907a8c"
Oct 30 19:48:43.614928 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.614901    4524 generic.go:356] "Generic (PLEG): container finished" podID="3689eb31-fbba-4cbf-a355-7056ccec100b" containerID="7080939e88c720ab0317423c72842b71ecd72a8f429dfbef10c6fb5341766ff9" exitCode=0
Oct 30 19:48:43.615054 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.614946    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh"
Oct 30 19:48:43.615054 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.615007    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh" event={"ID":"3689eb31-fbba-4cbf-a355-7056ccec100b","Type":"ContainerDied","Data":"7080939e88c720ab0317423c72842b71ecd72a8f429dfbef10c6fb5341766ff9"}
Oct 30 19:48:43.615203 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.615079    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh" event={"ID":"3689eb31-fbba-4cbf-a355-7056ccec100b","Type":"ContainerDied","Data":"a01728676bb46d5df816bd641a4878ec839ef0a00bd60b23ab05c943e5c8a2a9"}
Oct 30 19:48:43.619008 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.618985    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs" event={"ID":"6fb90c94-7e88-4cd8-96c1-f779013b1694","Type":"ContainerDied","Data":"b1b751a8c9ce625bbe0b55a21d411c2428f840160d00a51fb48f1e6ed2740c39"}
Oct 30 19:48:43.619101 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.619071    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"
Oct 30 19:48:43.638537 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.626020    4524 scope.go:122] "RemoveContainer" containerID="e866f74e712d90cf48a0f818e0cdfdcbee70f49fab1722a1963d019b54907a8c"
Oct 30 19:48:43.638537 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:48:43.626315    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e866f74e712d90cf48a0f818e0cdfdcbee70f49fab1722a1963d019b54907a8c\": container with ID starting with e866f74e712d90cf48a0f818e0cdfdcbee70f49fab1722a1963d019b54907a8c not found: ID does not exist" containerID="e866f74e712d90cf48a0f818e0cdfdcbee70f49fab1722a1963d019b54907a8c"
Oct 30 19:48:43.638537 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.626347    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e866f74e712d90cf48a0f818e0cdfdcbee70f49fab1722a1963d019b54907a8c"} err="failed to get container status \"e866f74e712d90cf48a0f818e0cdfdcbee70f49fab1722a1963d019b54907a8c\": rpc error: code = NotFound desc = could not find container \"e866f74e712d90cf48a0f818e0cdfdcbee70f49fab1722a1963d019b54907a8c\": container with ID starting with e866f74e712d90cf48a0f818e0cdfdcbee70f49fab1722a1963d019b54907a8c not found: ID does not exist"
Oct 30 19:48:43.638537 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.626367    4524 scope.go:122] "RemoveContainer" containerID="7080939e88c720ab0317423c72842b71ecd72a8f429dfbef10c6fb5341766ff9"
Oct 30 19:48:43.655881 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.655847    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq"]
Oct 30 19:48:43.660543 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.660509    4524 scope.go:122] "RemoveContainer" containerID="7080939e88c720ab0317423c72842b71ecd72a8f429dfbef10c6fb5341766ff9"
Oct 30 19:48:43.662011 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:48:43.661004    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7080939e88c720ab0317423c72842b71ecd72a8f429dfbef10c6fb5341766ff9\": container with ID starting with 7080939e88c720ab0317423c72842b71ecd72a8f429dfbef10c6fb5341766ff9 not found: ID does not exist" containerID="7080939e88c720ab0317423c72842b71ecd72a8f429dfbef10c6fb5341766ff9"
Oct 30 19:48:43.662011 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.661045    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7080939e88c720ab0317423c72842b71ecd72a8f429dfbef10c6fb5341766ff9"} err="failed to get container status \"7080939e88c720ab0317423c72842b71ecd72a8f429dfbef10c6fb5341766ff9\": rpc error: code = NotFound desc = could not find container \"7080939e88c720ab0317423c72842b71ecd72a8f429dfbef10c6fb5341766ff9\": container with ID starting with 7080939e88c720ab0317423c72842b71ecd72a8f429dfbef10c6fb5341766ff9 not found: ID does not exist"
Oct 30 19:48:43.662011 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.661074    4524 scope.go:122] "RemoveContainer" containerID="67f952639a0536feff2cb15669432494070cf290a293463f09a54fc9d5c5ed32"
Oct 30 19:48:43.662989 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.662962    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-2d5d53-6c4785ff8b-9xswq"]
Oct 30 19:48:43.674568 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.674548    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh"]
Oct 30 19:48:43.682122 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.682101    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-2d5d53-68f776f4ff-5wwsh"]
Oct 30 19:48:43.694947 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.694927    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"]
Oct 30 19:48:43.698835 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.698812    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ddb6ea-784666b5cb-kf6xs"]
Oct 30 19:48:43.922078 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.921997    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"]
Oct 30 19:48:43.922583 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:43.922550    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd" podUID="e53dd061-8aa9-4ace-9140-aea8a7375c20" containerName="istio-proxy" containerID="cri-o://64fa6bef4c93bcee559a775b47195fcbcdf08108406e3c784e0391d901a5af49" gracePeriod=30
Oct 30 19:48:44.392139 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:44.392103    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1affe672-c981-43ad-bd2c-099648e4be4c" path="/var/lib/kubelet/pods/1affe672-c981-43ad-bd2c-099648e4be4c/volumes"
Oct 30 19:48:44.392785 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:44.392749    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3689eb31-fbba-4cbf-a355-7056ccec100b" path="/var/lib/kubelet/pods/3689eb31-fbba-4cbf-a355-7056ccec100b/volumes"
Oct 30 19:48:44.393342 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:44.393323    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6fb90c94-7e88-4cd8-96c1-f779013b1694" path="/var/lib/kubelet/pods/6fb90c94-7e88-4cd8-96c1-f779013b1694/volumes"
Oct 30 19:48:44.394342 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:44.394317    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ac2f9eb-a945-483a-9c0a-7ad91b8fd618" path="/var/lib/kubelet/pods/7ac2f9eb-a945-483a-9c0a-7ad91b8fd618/volumes"
Oct 30 19:48:44.394856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:44.394836    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db66fd6a-afe9-47e7-815c-5c66c86081d8" path="/var/lib/kubelet/pods/db66fd6a-afe9-47e7-815c-5c66c86081d8/volumes"
Oct 30 19:48:45.625780 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:48:45.625736    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:48:49.237032 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.237001    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:48:49.333274 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.333243    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-token\" (UniqueName: \"kubernetes.io/projected/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-token\") pod \"e53dd061-8aa9-4ace-9140-aea8a7375c20\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") "
Oct 30 19:48:49.333457 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.333287    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-credential-socket\") pod \"e53dd061-8aa9-4ace-9140-aea8a7375c20\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") "
Oct 30 19:48:49.333457 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.333346    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-envoy\") pod \"e53dd061-8aa9-4ace-9140-aea8a7375c20\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") "
Oct 30 19:48:49.333457 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.333370    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-workload-socket\") pod \"e53dd061-8aa9-4ace-9140-aea8a7375c20\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") "
Oct 30 19:48:49.333457 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.333413    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-data\") pod \"e53dd061-8aa9-4ace-9140-aea8a7375c20\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") "
Oct 30 19:48:49.333457 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.333451    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-podinfo\") pod \"e53dd061-8aa9-4ace-9140-aea8a7375c20\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") "
Oct 30 19:48:49.333809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.333522    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e53dd061-8aa9-4ace-9140-aea8a7375c20-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e53dd061-8aa9-4ace-9140-aea8a7375c20-istiod-ca-cert\") pod \"e53dd061-8aa9-4ace-9140-aea8a7375c20\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") "
Oct 30 19:48:49.333809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.333545    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-workload-certs\") pod \"e53dd061-8aa9-4ace-9140-aea8a7375c20\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") "
Oct 30 19:48:49.333809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.333571    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e53dd061-8aa9-4ace-9140-aea8a7375c20-kube-api-access-5wk2v\" (UniqueName: \"kubernetes.io/projected/e53dd061-8aa9-4ace-9140-aea8a7375c20-kube-api-access-5wk2v\") pod \"e53dd061-8aa9-4ace-9140-aea8a7375c20\" (UID: \"e53dd061-8aa9-4ace-9140-aea8a7375c20\") "
Oct 30 19:48:49.333809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.333692    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-credential-socket" pod "e53dd061-8aa9-4ace-9140-aea8a7375c20" (UID: "e53dd061-8aa9-4ace-9140-aea8a7375c20"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:49.333809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.333775    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-data" pod "e53dd061-8aa9-4ace-9140-aea8a7375c20" (UID: "e53dd061-8aa9-4ace-9140-aea8a7375c20"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:49.333809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.333684    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-workload-socket" pod "e53dd061-8aa9-4ace-9140-aea8a7375c20" (UID: "e53dd061-8aa9-4ace-9140-aea8a7375c20"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:49.334105 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.333879    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-workload-certs" pod "e53dd061-8aa9-4ace-9140-aea8a7375c20" (UID: "e53dd061-8aa9-4ace-9140-aea8a7375c20"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:49.334105 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.334054    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:49.334105 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.334077    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:49.334105 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.334089    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:49.334105 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.334098    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:49.334316 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.334291    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e53dd061-8aa9-4ace-9140-aea8a7375c20-istiod-ca-cert" pod "e53dd061-8aa9-4ace-9140-aea8a7375c20" (UID: "e53dd061-8aa9-4ace-9140-aea8a7375c20"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:48:49.335784 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.335758    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-podinfo" pod "e53dd061-8aa9-4ace-9140-aea8a7375c20" (UID: "e53dd061-8aa9-4ace-9140-aea8a7375c20"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:48:49.335896 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.335834    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e53dd061-8aa9-4ace-9140-aea8a7375c20-kube-api-access-5wk2v" pod "e53dd061-8aa9-4ace-9140-aea8a7375c20" (UID: "e53dd061-8aa9-4ace-9140-aea8a7375c20"). InnerVolumeSpecName "kube-api-access-5wk2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:49.335994 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.335977    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-token" pod "e53dd061-8aa9-4ace-9140-aea8a7375c20" (UID: "e53dd061-8aa9-4ace-9140-aea8a7375c20"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:48:49.336054 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.335975    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-envoy" pod "e53dd061-8aa9-4ace-9140-aea8a7375c20" (UID: "e53dd061-8aa9-4ace-9140-aea8a7375c20"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:48:49.435385 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.435357    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:49.435385 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.435379    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:49.435385 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.435390    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e53dd061-8aa9-4ace-9140-aea8a7375c20-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:49.435628 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.435399    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wk2v\" (UniqueName: \"kubernetes.io/projected/e53dd061-8aa9-4ace-9140-aea8a7375c20-kube-api-access-5wk2v\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:49.435628 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.435413    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e53dd061-8aa9-4ace-9140-aea8a7375c20-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:48:49.714078 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.714041    4524 generic.go:356] "Generic (PLEG): container finished" podID="e53dd061-8aa9-4ace-9140-aea8a7375c20" containerID="64fa6bef4c93bcee559a775b47195fcbcdf08108406e3c784e0391d901a5af49" exitCode=0
Oct 30 19:48:49.714302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.714098    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"
Oct 30 19:48:49.714302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.714130    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd" event={"ID":"e53dd061-8aa9-4ace-9140-aea8a7375c20","Type":"ContainerDied","Data":"64fa6bef4c93bcee559a775b47195fcbcdf08108406e3c784e0391d901a5af49"}
Oct 30 19:48:49.714302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.714216    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd" event={"ID":"e53dd061-8aa9-4ace-9140-aea8a7375c20","Type":"ContainerDied","Data":"2e07d8c7624dfb06fcd06adb70f4809c834cd6e6bb45f711a8720fbfae241596"}
Oct 30 19:48:49.714302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.714240    4524 scope.go:122] "RemoveContainer" containerID="64fa6bef4c93bcee559a775b47195fcbcdf08108406e3c784e0391d901a5af49"
Oct 30 19:48:49.725970 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.725946    4524 scope.go:122] "RemoveContainer" containerID="64fa6bef4c93bcee559a775b47195fcbcdf08108406e3c784e0391d901a5af49"
Oct 30 19:48:49.726270 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:48:49.726253    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64fa6bef4c93bcee559a775b47195fcbcdf08108406e3c784e0391d901a5af49\": container with ID starting with 64fa6bef4c93bcee559a775b47195fcbcdf08108406e3c784e0391d901a5af49 not found: ID does not exist" containerID="64fa6bef4c93bcee559a775b47195fcbcdf08108406e3c784e0391d901a5af49"
Oct 30 19:48:49.726340 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.726279    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64fa6bef4c93bcee559a775b47195fcbcdf08108406e3c784e0391d901a5af49"} err="failed to get container status \"64fa6bef4c93bcee559a775b47195fcbcdf08108406e3c784e0391d901a5af49\": rpc error: code = NotFound desc = could not find container \"64fa6bef4c93bcee559a775b47195fcbcdf08108406e3c784e0391d901a5af49\": container with ID starting with 64fa6bef4c93bcee559a775b47195fcbcdf08108406e3c784e0391d901a5af49 not found: ID does not exist"
Oct 30 19:48:49.802272 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.802241    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"]
Oct 30 19:48:49.823094 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:49.823065    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-2d5d53-openshift-default-6dd7664849-qxqqd"]
Oct 30 19:48:50.390032 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:48:50.389961    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e53dd061-8aa9-4ace-9140-aea8a7375c20" path="/var/lib/kubelet/pods/e53dd061-8aa9-4ace-9140-aea8a7375c20/volumes"
Oct 30 19:49:04.544049 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:04.544009    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9"]
Oct 30 19:49:05.595264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:05.595231    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"]
Oct 30 19:49:05.595781 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:05.595520    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh" podUID="914487e6-51d5-4f89-9aa7-e6cd7f329a3e" containerName="maas-api" containerID="cri-o://b8fb7132dccd04e2758ce517fa52b112532fdd2c84a814584c700c7e2b9bcab3" gracePeriod=30
Oct 30 19:49:05.832494 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:05.832273    4524 generic.go:356] "Generic (PLEG): container finished" podID="914487e6-51d5-4f89-9aa7-e6cd7f329a3e" containerID="b8fb7132dccd04e2758ce517fa52b112532fdd2c84a814584c700c7e2b9bcab3" exitCode=0
Oct 30 19:49:05.832494 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:05.832426    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh" event={"ID":"914487e6-51d5-4f89-9aa7-e6cd7f329a3e","Type":"ContainerDied","Data":"b8fb7132dccd04e2758ce517fa52b112532fdd2c84a814584c700c7e2b9bcab3"}
Oct 30 19:49:05.937801 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:05.937777    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"
Oct 30 19:49:06.146537 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:06.145585    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-metrics-certs\") pod \"914487e6-51d5-4f89-9aa7-e6cd7f329a3e\" (UID: \"914487e6-51d5-4f89-9aa7-e6cd7f329a3e\") "
Oct 30 19:49:06.146537 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:06.145739    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-kube-api-access-8phnv\" (UniqueName: \"kubernetes.io/projected/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-kube-api-access-8phnv\") pod \"914487e6-51d5-4f89-9aa7-e6cd7f329a3e\" (UID: \"914487e6-51d5-4f89-9aa7-e6cd7f329a3e\") "
Oct 30 19:49:06.146537 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:06.145794    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-maas-api-tls\") pod \"914487e6-51d5-4f89-9aa7-e6cd7f329a3e\" (UID: \"914487e6-51d5-4f89-9aa7-e6cd7f329a3e\") "
Oct 30 19:49:06.146537 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:06.145904    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-openshift-service-ca\") pod \"914487e6-51d5-4f89-9aa7-e6cd7f329a3e\" (UID: \"914487e6-51d5-4f89-9aa7-e6cd7f329a3e\") "
Oct 30 19:49:06.147132 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:06.147105    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-openshift-service-ca" pod "914487e6-51d5-4f89-9aa7-e6cd7f329a3e" (UID: "914487e6-51d5-4f89-9aa7-e6cd7f329a3e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:06.158490 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:06.158452    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-metrics-certs" pod "914487e6-51d5-4f89-9aa7-e6cd7f329a3e" (UID: "914487e6-51d5-4f89-9aa7-e6cd7f329a3e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:49:06.161348 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:06.161306    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-maas-api-tls" pod "914487e6-51d5-4f89-9aa7-e6cd7f329a3e" (UID: "914487e6-51d5-4f89-9aa7-e6cd7f329a3e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:49:06.161488 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:06.161340    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-kube-api-access-8phnv" pod "914487e6-51d5-4f89-9aa7-e6cd7f329a3e" (UID: "914487e6-51d5-4f89-9aa7-e6cd7f329a3e"). InnerVolumeSpecName "kube-api-access-8phnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:06.247659 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:06.247618    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8phnv\" (UniqueName: \"kubernetes.io/projected/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-kube-api-access-8phnv\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:06.247851 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:06.247673    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:06.247851 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:06.247684    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:06.247851 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:06.247693    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/914487e6-51d5-4f89-9aa7-e6cd7f329a3e-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:06.842971 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:06.842941    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"
Oct 30 19:49:06.843453 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:06.842969    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh" event={"ID":"914487e6-51d5-4f89-9aa7-e6cd7f329a3e","Type":"ContainerDied","Data":"2fa7e34e104834ab4356ef2178e8ebfe25d3589c35f54987620362388ff658ab"}
Oct 30 19:49:06.843453 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:06.843034    4524 scope.go:122] "RemoveContainer" containerID="b8fb7132dccd04e2758ce517fa52b112532fdd2c84a814584c700c7e2b9bcab3"
Oct 30 19:49:06.861311 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:06.861287    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"]
Oct 30 19:49:06.867450 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:06.867428    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-fd7c77-76c86f59c7-pjmsh"]
Oct 30 19:49:08.389614 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:08.389583    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="914487e6-51d5-4f89-9aa7-e6cd7f329a3e" path="/var/lib/kubelet/pods/914487e6-51d5-4f89-9aa7-e6cd7f329a3e/volumes"
Oct 30 19:49:09.544802 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:09.544763    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9" podUID="7ad49a59-872d-48c0-91f7-84a249089008" containerName="payload-pre-processing" containerID="cri-o://086886a9bcb562dc32ce05dddcea0e7432333be1733158f058277c06338aae9e" gracePeriod=25
Oct 30 19:49:13.285993 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.285956    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"]
Oct 30 19:49:13.286513 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.286213    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9" podUID="68c74fd5-f30e-4eb7-bc88-e2b679a28451" containerName="maas-api" containerID="cri-o://eb88dac7d0077e079f9e895199efa5af9e6a4285a5dee8aa98b842c92b34791a" gracePeriod=30
Oct 30 19:49:13.639896 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.639866    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"
Oct 30 19:49:13.743979 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.743946    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/68c74fd5-f30e-4eb7-bc88-e2b679a28451-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68c74fd5-f30e-4eb7-bc88-e2b679a28451-openshift-service-ca\") pod \"68c74fd5-f30e-4eb7-bc88-e2b679a28451\" (UID: \"68c74fd5-f30e-4eb7-bc88-e2b679a28451\") "
Oct 30 19:49:13.744142 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.743993    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/68c74fd5-f30e-4eb7-bc88-e2b679a28451-kube-api-access-hk22c\" (UniqueName: \"kubernetes.io/projected/68c74fd5-f30e-4eb7-bc88-e2b679a28451-kube-api-access-hk22c\") pod \"68c74fd5-f30e-4eb7-bc88-e2b679a28451\" (UID: \"68c74fd5-f30e-4eb7-bc88-e2b679a28451\") "
Oct 30 19:49:13.744142 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.744018    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/68c74fd5-f30e-4eb7-bc88-e2b679a28451-metrics-certs\" (UniqueName: \"kubernetes.io/secret/68c74fd5-f30e-4eb7-bc88-e2b679a28451-metrics-certs\") pod \"68c74fd5-f30e-4eb7-bc88-e2b679a28451\" (UID: \"68c74fd5-f30e-4eb7-bc88-e2b679a28451\") "
Oct 30 19:49:13.744302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.744198    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/68c74fd5-f30e-4eb7-bc88-e2b679a28451-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68c74fd5-f30e-4eb7-bc88-e2b679a28451-maas-api-tls\") pod \"68c74fd5-f30e-4eb7-bc88-e2b679a28451\" (UID: \"68c74fd5-f30e-4eb7-bc88-e2b679a28451\") "
Oct 30 19:49:13.744373 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.744329    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68c74fd5-f30e-4eb7-bc88-e2b679a28451-openshift-service-ca" pod "68c74fd5-f30e-4eb7-bc88-e2b679a28451" (UID: "68c74fd5-f30e-4eb7-bc88-e2b679a28451"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:13.744619 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.744596    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/68c74fd5-f30e-4eb7-bc88-e2b679a28451-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:13.746126 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.746100    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68c74fd5-f30e-4eb7-bc88-e2b679a28451-maas-api-tls" pod "68c74fd5-f30e-4eb7-bc88-e2b679a28451" (UID: "68c74fd5-f30e-4eb7-bc88-e2b679a28451"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:49:13.746271 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.746200    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68c74fd5-f30e-4eb7-bc88-e2b679a28451-kube-api-access-hk22c" pod "68c74fd5-f30e-4eb7-bc88-e2b679a28451" (UID: "68c74fd5-f30e-4eb7-bc88-e2b679a28451"). InnerVolumeSpecName "kube-api-access-hk22c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:13.746271 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.746208    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68c74fd5-f30e-4eb7-bc88-e2b679a28451-metrics-certs" pod "68c74fd5-f30e-4eb7-bc88-e2b679a28451" (UID: "68c74fd5-f30e-4eb7-bc88-e2b679a28451"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:49:13.860237 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.854014    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hk22c\" (UniqueName: \"kubernetes.io/projected/68c74fd5-f30e-4eb7-bc88-e2b679a28451-kube-api-access-hk22c\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:13.860237 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.854052    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/68c74fd5-f30e-4eb7-bc88-e2b679a28451-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:13.860237 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.854066    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/68c74fd5-f30e-4eb7-bc88-e2b679a28451-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:13.892635 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.892555    4524 generic.go:356] "Generic (PLEG): container finished" podID="68c74fd5-f30e-4eb7-bc88-e2b679a28451" containerID="eb88dac7d0077e079f9e895199efa5af9e6a4285a5dee8aa98b842c92b34791a" exitCode=0
Oct 30 19:49:13.892635 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.892615    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"
Oct 30 19:49:13.892826 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.892644    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9" event={"ID":"68c74fd5-f30e-4eb7-bc88-e2b679a28451","Type":"ContainerDied","Data":"eb88dac7d0077e079f9e895199efa5af9e6a4285a5dee8aa98b842c92b34791a"}
Oct 30 19:49:13.892826 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.892706    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9" event={"ID":"68c74fd5-f30e-4eb7-bc88-e2b679a28451","Type":"ContainerDied","Data":"267b89d84b7b78fddb58d3a5813f60018b39a932223d58480b609aa66e3b3659"}
Oct 30 19:49:13.892826 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.892737    4524 scope.go:122] "RemoveContainer" containerID="eb88dac7d0077e079f9e895199efa5af9e6a4285a5dee8aa98b842c92b34791a"
Oct 30 19:49:13.904949 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.904922    4524 scope.go:122] "RemoveContainer" containerID="eb88dac7d0077e079f9e895199efa5af9e6a4285a5dee8aa98b842c92b34791a"
Oct 30 19:49:13.905286 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:49:13.905266    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb88dac7d0077e079f9e895199efa5af9e6a4285a5dee8aa98b842c92b34791a\": container with ID starting with eb88dac7d0077e079f9e895199efa5af9e6a4285a5dee8aa98b842c92b34791a not found: ID does not exist" containerID="eb88dac7d0077e079f9e895199efa5af9e6a4285a5dee8aa98b842c92b34791a"
Oct 30 19:49:13.905391 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.905296    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb88dac7d0077e079f9e895199efa5af9e6a4285a5dee8aa98b842c92b34791a"} err="failed to get container status \"eb88dac7d0077e079f9e895199efa5af9e6a4285a5dee8aa98b842c92b34791a\": rpc error: code = NotFound desc = could not find container \"eb88dac7d0077e079f9e895199efa5af9e6a4285a5dee8aa98b842c92b34791a\": container with ID starting with eb88dac7d0077e079f9e895199efa5af9e6a4285a5dee8aa98b842c92b34791a not found: ID does not exist"
Oct 30 19:49:13.917982 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.917957    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"]
Oct 30 19:49:13.921331 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:13.921305    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-39b867-7bc8b889b7-hvfx9"]
Oct 30 19:49:14.395141 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:14.395097    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="68c74fd5-f30e-4eb7-bc88-e2b679a28451" path="/var/lib/kubelet/pods/68c74fd5-f30e-4eb7-bc88-e2b679a28451/volumes"
Oct 30 19:49:14.443819 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:14.443783    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"]
Oct 30 19:49:14.444061 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:14.444031    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d" podUID="fb3f6371-c4db-4bd9-8101-b34064bb6cc7" containerName="istio-proxy" containerID="cri-o://20ffa972d1f2f04cfbff37644ef557c1cd6e46d135c88b4abcc252d7844e74a4" gracePeriod=30
Oct 30 19:49:15.627591 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:49:15.627555    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:49:18.699109 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:18.699059    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9" podUID="7ad49a59-872d-48c0-91f7-84a249089008" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:49:19.718939 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.718876    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:49:19.818551 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.818513    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-envoy\") pod \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") "
Oct 30 19:49:19.818764 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.818571    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-token\" (UniqueName: \"kubernetes.io/projected/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-token\") pod \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") "
Oct 30 19:49:19.818764 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.818598    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istiod-ca-cert\") pod \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") "
Oct 30 19:49:19.818764 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.818630    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-workload-certs\") pod \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") "
Oct 30 19:49:19.818764 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.818692    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-workload-socket\") pod \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") "
Oct 30 19:49:19.818764 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.818733    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-data\") pod \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") "
Oct 30 19:49:19.819074 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.818778    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-credential-socket\") pod \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") "
Oct 30 19:49:19.819074 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.818812    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-kube-api-access-ngq7c\" (UniqueName: \"kubernetes.io/projected/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-kube-api-access-ngq7c\") pod \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") "
Oct 30 19:49:19.819074 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.818854    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-podinfo\") pod \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\" (UID: \"fb3f6371-c4db-4bd9-8101-b34064bb6cc7\") "
Oct 30 19:49:19.819402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.819372    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-workload-certs" pod "fb3f6371-c4db-4bd9-8101-b34064bb6cc7" (UID: "fb3f6371-c4db-4bd9-8101-b34064bb6cc7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:19.819771 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.819687    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-workload-socket" pod "fb3f6371-c4db-4bd9-8101-b34064bb6cc7" (UID: "fb3f6371-c4db-4bd9-8101-b34064bb6cc7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:19.819891 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.819809    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istiod-ca-cert" pod "fb3f6371-c4db-4bd9-8101-b34064bb6cc7" (UID: "fb3f6371-c4db-4bd9-8101-b34064bb6cc7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:19.819973 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.819893    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-data" pod "fb3f6371-c4db-4bd9-8101-b34064bb6cc7" (UID: "fb3f6371-c4db-4bd9-8101-b34064bb6cc7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:19.820805 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.820309    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-credential-socket" pod "fb3f6371-c4db-4bd9-8101-b34064bb6cc7" (UID: "fb3f6371-c4db-4bd9-8101-b34064bb6cc7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:19.822153 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.822125    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-token" pod "fb3f6371-c4db-4bd9-8101-b34064bb6cc7" (UID: "fb3f6371-c4db-4bd9-8101-b34064bb6cc7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:19.822526 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.822436    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-kube-api-access-ngq7c" pod "fb3f6371-c4db-4bd9-8101-b34064bb6cc7" (UID: "fb3f6371-c4db-4bd9-8101-b34064bb6cc7"). InnerVolumeSpecName "kube-api-access-ngq7c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:19.823581 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.823552    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-envoy" pod "fb3f6371-c4db-4bd9-8101-b34064bb6cc7" (UID: "fb3f6371-c4db-4bd9-8101-b34064bb6cc7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:19.824539 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.824517    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-podinfo" pod "fb3f6371-c4db-4bd9-8101-b34064bb6cc7" (UID: "fb3f6371-c4db-4bd9-8101-b34064bb6cc7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:49:19.920817 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.920782    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:19.920817 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.920816    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:19.921054 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.920829    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:19.921054 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.920841    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:19.921054 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.920859    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:19.921054 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.920872    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngq7c\" (UniqueName: \"kubernetes.io/projected/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-kube-api-access-ngq7c\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:19.921054 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.920885    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:19.921054 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.920898    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:19.921054 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.920909    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb3f6371-c4db-4bd9-8101-b34064bb6cc7-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:19.950779 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.950744    4524 generic.go:356] "Generic (PLEG): container finished" podID="fb3f6371-c4db-4bd9-8101-b34064bb6cc7" containerID="20ffa972d1f2f04cfbff37644ef557c1cd6e46d135c88b4abcc252d7844e74a4" exitCode=0
Oct 30 19:49:19.950906 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.950798    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"
Oct 30 19:49:19.950906 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.950829    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d" event={"ID":"fb3f6371-c4db-4bd9-8101-b34064bb6cc7","Type":"ContainerDied","Data":"20ffa972d1f2f04cfbff37644ef557c1cd6e46d135c88b4abcc252d7844e74a4"}
Oct 30 19:49:19.950906 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.950884    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d" event={"ID":"fb3f6371-c4db-4bd9-8101-b34064bb6cc7","Type":"ContainerDied","Data":"e545117a2cb28a7c056aa7d60a7e32095e94fbcee9d194cce7b3c119f7157e8e"}
Oct 30 19:49:19.951019 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.950907    4524 scope.go:122] "RemoveContainer" containerID="20ffa972d1f2f04cfbff37644ef557c1cd6e46d135c88b4abcc252d7844e74a4"
Oct 30 19:49:19.961264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.961228    4524 scope.go:122] "RemoveContainer" containerID="20ffa972d1f2f04cfbff37644ef557c1cd6e46d135c88b4abcc252d7844e74a4"
Oct 30 19:49:19.961569 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:49:19.961544    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20ffa972d1f2f04cfbff37644ef557c1cd6e46d135c88b4abcc252d7844e74a4\": container with ID starting with 20ffa972d1f2f04cfbff37644ef557c1cd6e46d135c88b4abcc252d7844e74a4 not found: ID does not exist" containerID="20ffa972d1f2f04cfbff37644ef557c1cd6e46d135c88b4abcc252d7844e74a4"
Oct 30 19:49:19.961609 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.961583    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20ffa972d1f2f04cfbff37644ef557c1cd6e46d135c88b4abcc252d7844e74a4"} err="failed to get container status \"20ffa972d1f2f04cfbff37644ef557c1cd6e46d135c88b4abcc252d7844e74a4\": rpc error: code = NotFound desc = could not find container \"20ffa972d1f2f04cfbff37644ef557c1cd6e46d135c88b4abcc252d7844e74a4\": container with ID starting with 20ffa972d1f2f04cfbff37644ef557c1cd6e46d135c88b4abcc252d7844e74a4 not found: ID does not exist"
Oct 30 19:49:19.973092 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.973039    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"]
Oct 30 19:49:19.978799 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:19.978777    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-fd7c77-openshift-default-6995885647-kdw8d"]
Oct 30 19:49:20.388956 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:20.388925    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb3f6371-c4db-4bd9-8101-b34064bb6cc7" path="/var/lib/kubelet/pods/fb3f6371-c4db-4bd9-8101-b34064bb6cc7/volumes"
Oct 30 19:49:22.277193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:22.276629    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"]
Oct 30 19:49:22.277193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:22.276909    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf" podUID="c0210704-8d10-4dad-8f61-7e028f2bbab8" containerName="istio-proxy" containerID="cri-o://44a801cc581e21f3201f5f06290ac56ef7bbcf9d7acf76986d3c87e3628c09ee" gracePeriod=30
Oct 30 19:49:25.218922 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:25.218884    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8"]
Oct 30 19:49:25.229486 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:25.229458    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd"]
Oct 30 19:49:27.946800 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:27.946772    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:49:28.001103 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.001064    4524 generic.go:356] "Generic (PLEG): container finished" podID="c0210704-8d10-4dad-8f61-7e028f2bbab8" containerID="44a801cc581e21f3201f5f06290ac56ef7bbcf9d7acf76986d3c87e3628c09ee" exitCode=0
Oct 30 19:49:28.001287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.001103    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"
Oct 30 19:49:28.001287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.001142    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf" event={"ID":"c0210704-8d10-4dad-8f61-7e028f2bbab8","Type":"ContainerDied","Data":"44a801cc581e21f3201f5f06290ac56ef7bbcf9d7acf76986d3c87e3628c09ee"}
Oct 30 19:49:28.001287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.001220    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf" event={"ID":"c0210704-8d10-4dad-8f61-7e028f2bbab8","Type":"ContainerDied","Data":"b87110d1d94d5e99a8fc03b85f58cfa9fbed41caaa56ce51b80cc9a40a6b789e"}
Oct 30 19:49:28.001287 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.001256    4524 scope.go:122] "RemoveContainer" containerID="44a801cc581e21f3201f5f06290ac56ef7bbcf9d7acf76986d3c87e3628c09ee"
Oct 30 19:49:28.004743 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.004721    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-credential-socket\") pod \"c0210704-8d10-4dad-8f61-7e028f2bbab8\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") "
Oct 30 19:49:28.004864 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.004770    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0210704-8d10-4dad-8f61-7e028f2bbab8-kube-api-access-r45dm\" (UniqueName: \"kubernetes.io/projected/c0210704-8d10-4dad-8f61-7e028f2bbab8-kube-api-access-r45dm\") pod \"c0210704-8d10-4dad-8f61-7e028f2bbab8\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") "
Oct 30 19:49:28.004864 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.004799    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-workload-certs\") pod \"c0210704-8d10-4dad-8f61-7e028f2bbab8\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") "
Oct 30 19:49:28.004864 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.004838    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-podinfo\") pod \"c0210704-8d10-4dad-8f61-7e028f2bbab8\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") "
Oct 30 19:49:28.005049 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.004875    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c0210704-8d10-4dad-8f61-7e028f2bbab8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0210704-8d10-4dad-8f61-7e028f2bbab8-istiod-ca-cert\") pod \"c0210704-8d10-4dad-8f61-7e028f2bbab8\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") "
Oct 30 19:49:28.005049 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.005024    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-credential-socket" pod "c0210704-8d10-4dad-8f61-7e028f2bbab8" (UID: "c0210704-8d10-4dad-8f61-7e028f2bbab8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:28.005164 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.005144    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-token\" (UniqueName: \"kubernetes.io/projected/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-token\") pod \"c0210704-8d10-4dad-8f61-7e028f2bbab8\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") "
Oct 30 19:49:28.005270 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.005153    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-workload-certs" pod "c0210704-8d10-4dad-8f61-7e028f2bbab8" (UID: "c0210704-8d10-4dad-8f61-7e028f2bbab8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:28.005270 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.005218    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-data\") pod \"c0210704-8d10-4dad-8f61-7e028f2bbab8\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") "
Oct 30 19:49:28.005270 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.005243    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0210704-8d10-4dad-8f61-7e028f2bbab8-istiod-ca-cert" pod "c0210704-8d10-4dad-8f61-7e028f2bbab8" (UID: "c0210704-8d10-4dad-8f61-7e028f2bbab8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:28.005439 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.005280    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-envoy\") pod \"c0210704-8d10-4dad-8f61-7e028f2bbab8\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") "
Oct 30 19:49:28.005439 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.005364    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-workload-socket\") pod \"c0210704-8d10-4dad-8f61-7e028f2bbab8\" (UID: \"c0210704-8d10-4dad-8f61-7e028f2bbab8\") "
Oct 30 19:49:28.005777 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.005456    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-data" pod "c0210704-8d10-4dad-8f61-7e028f2bbab8" (UID: "c0210704-8d10-4dad-8f61-7e028f2bbab8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:28.005777 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.005730    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-workload-socket" pod "c0210704-8d10-4dad-8f61-7e028f2bbab8" (UID: "c0210704-8d10-4dad-8f61-7e028f2bbab8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:28.005777 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.005739    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:28.005777 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.005762    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:28.005777 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.005779    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0210704-8d10-4dad-8f61-7e028f2bbab8-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:28.005982 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.005793    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:28.007435 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.007411    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0210704-8d10-4dad-8f61-7e028f2bbab8-kube-api-access-r45dm" pod "c0210704-8d10-4dad-8f61-7e028f2bbab8" (UID: "c0210704-8d10-4dad-8f61-7e028f2bbab8"). InnerVolumeSpecName "kube-api-access-r45dm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:28.007654 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.007623    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-token" pod "c0210704-8d10-4dad-8f61-7e028f2bbab8" (UID: "c0210704-8d10-4dad-8f61-7e028f2bbab8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:28.007753 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.007653    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-envoy" pod "c0210704-8d10-4dad-8f61-7e028f2bbab8" (UID: "c0210704-8d10-4dad-8f61-7e028f2bbab8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:28.008159 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.008143    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-podinfo" pod "c0210704-8d10-4dad-8f61-7e028f2bbab8" (UID: "c0210704-8d10-4dad-8f61-7e028f2bbab8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:49:28.013424 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.013406    4524 scope.go:122] "RemoveContainer" containerID="44a801cc581e21f3201f5f06290ac56ef7bbcf9d7acf76986d3c87e3628c09ee"
Oct 30 19:49:28.013702 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:49:28.013684    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44a801cc581e21f3201f5f06290ac56ef7bbcf9d7acf76986d3c87e3628c09ee\": container with ID starting with 44a801cc581e21f3201f5f06290ac56ef7bbcf9d7acf76986d3c87e3628c09ee not found: ID does not exist" containerID="44a801cc581e21f3201f5f06290ac56ef7bbcf9d7acf76986d3c87e3628c09ee"
Oct 30 19:49:28.013762 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.013714    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44a801cc581e21f3201f5f06290ac56ef7bbcf9d7acf76986d3c87e3628c09ee"} err="failed to get container status \"44a801cc581e21f3201f5f06290ac56ef7bbcf9d7acf76986d3c87e3628c09ee\": rpc error: code = NotFound desc = could not find container \"44a801cc581e21f3201f5f06290ac56ef7bbcf9d7acf76986d3c87e3628c09ee\": container with ID starting with 44a801cc581e21f3201f5f06290ac56ef7bbcf9d7acf76986d3c87e3628c09ee not found: ID does not exist"
Oct 30 19:49:28.107082 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.107046    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:28.107082 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.107082    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:28.107082 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.107092    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0210704-8d10-4dad-8f61-7e028f2bbab8-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:28.107335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.107102    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r45dm\" (UniqueName: \"kubernetes.io/projected/c0210704-8d10-4dad-8f61-7e028f2bbab8-kube-api-access-r45dm\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:28.107335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.107112    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0210704-8d10-4dad-8f61-7e028f2bbab8-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:28.327139 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.327111    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"]
Oct 30 19:49:28.333306 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.333283    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-39b867-openshift-default-756f9d477f-7jjtf"]
Oct 30 19:49:28.389341 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.389287    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c0210704-8d10-4dad-8f61-7e028f2bbab8" path="/var/lib/kubelet/pods/c0210704-8d10-4dad-8f61-7e028f2bbab8/volumes"
Oct 30 19:49:28.698975 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:28.698889    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9" podUID="7ad49a59-872d-48c0-91f7-84a249089008" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:49:29.815925 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:29.815851    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9"
Oct 30 19:49:29.927667 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:29.927635    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ad49a59-872d-48c0-91f7-84a249089008-kube-api-access-kvj8h\" (UniqueName: \"kubernetes.io/projected/7ad49a59-872d-48c0-91f7-84a249089008-kube-api-access-kvj8h\") pod \"7ad49a59-872d-48c0-91f7-84a249089008\" (UID: \"7ad49a59-872d-48c0-91f7-84a249089008\") "
Oct 30 19:49:29.927863 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:29.927707    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ad49a59-872d-48c0-91f7-84a249089008-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ad49a59-872d-48c0-91f7-84a249089008-plugins-config-volume\") pod \"7ad49a59-872d-48c0-91f7-84a249089008\" (UID: \"7ad49a59-872d-48c0-91f7-84a249089008\") "
Oct 30 19:49:29.928181 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:29.928141    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ad49a59-872d-48c0-91f7-84a249089008-plugins-config-volume" pod "7ad49a59-872d-48c0-91f7-84a249089008" (UID: "7ad49a59-872d-48c0-91f7-84a249089008"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:29.929704 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:29.929681    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ad49a59-872d-48c0-91f7-84a249089008-kube-api-access-kvj8h" pod "7ad49a59-872d-48c0-91f7-84a249089008" (UID: "7ad49a59-872d-48c0-91f7-84a249089008"). InnerVolumeSpecName "kube-api-access-kvj8h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:30.017460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:30.017429    4524 generic.go:356] "Generic (PLEG): container finished" podID="7ad49a59-872d-48c0-91f7-84a249089008" containerID="086886a9bcb562dc32ce05dddcea0e7432333be1733158f058277c06338aae9e" exitCode=0
Oct 30 19:49:30.017579 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:30.017488    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9"
Oct 30 19:49:30.017579 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:30.017496    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9" event={"ID":"7ad49a59-872d-48c0-91f7-84a249089008","Type":"ContainerDied","Data":"086886a9bcb562dc32ce05dddcea0e7432333be1733158f058277c06338aae9e"}
Oct 30 19:49:30.017579 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:30.017570    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9" event={"ID":"7ad49a59-872d-48c0-91f7-84a249089008","Type":"ContainerDied","Data":"8d8d99a32b86ddcd18cb64a8e4afca46ef505c3d2a63c0fd0f2f62a6dc209501"}
Oct 30 19:49:30.017684 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:30.017594    4524 scope.go:122] "RemoveContainer" containerID="086886a9bcb562dc32ce05dddcea0e7432333be1733158f058277c06338aae9e"
Oct 30 19:49:30.027335 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:30.027249    4524 scope.go:122] "RemoveContainer" containerID="086886a9bcb562dc32ce05dddcea0e7432333be1733158f058277c06338aae9e"
Oct 30 19:49:30.027584 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:49:30.027555    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"086886a9bcb562dc32ce05dddcea0e7432333be1733158f058277c06338aae9e\": container with ID starting with 086886a9bcb562dc32ce05dddcea0e7432333be1733158f058277c06338aae9e not found: ID does not exist" containerID="086886a9bcb562dc32ce05dddcea0e7432333be1733158f058277c06338aae9e"
Oct 30 19:49:30.027659 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:30.027593    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"086886a9bcb562dc32ce05dddcea0e7432333be1733158f058277c06338aae9e"} err="failed to get container status \"086886a9bcb562dc32ce05dddcea0e7432333be1733158f058277c06338aae9e\": rpc error: code = NotFound desc = could not find container \"086886a9bcb562dc32ce05dddcea0e7432333be1733158f058277c06338aae9e\": container with ID starting with 086886a9bcb562dc32ce05dddcea0e7432333be1733158f058277c06338aae9e not found: ID does not exist"
Oct 30 19:49:30.028603 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:30.028583    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvj8h\" (UniqueName: \"kubernetes.io/projected/7ad49a59-872d-48c0-91f7-84a249089008-kube-api-access-kvj8h\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:30.028657 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:30.028607    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7ad49a59-872d-48c0-91f7-84a249089008-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:30.043493 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:30.043462    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9"]
Oct 30 19:49:30.046091 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:30.046067    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-fd7c77-79cc59d9cf-sjkh9"]
Oct 30 19:49:30.219887 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:30.219840    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8" podUID="8c861815-5c5d-4a6a-9916-fcedf26bc4e0" containerName="payload-processing" containerID="cri-o://0e9107cd32a24205c1758795c83c4b03b6a8ef3e9a365f5b5ed6c6af8d1b8158" gracePeriod=25
Oct 30 19:49:30.229797 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:30.229705    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd" podUID="fb610784-5f32-446f-9096-f524917ae3c7" containerName="payload-pre-processing" containerID="cri-o://f780342c2e4527fa0abfbb49e2590ad11a48c7c1a9365ce718fc687df7c5afe2" gracePeriod=25
Oct 30 19:49:30.389481 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:30.389438    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ad49a59-872d-48c0-91f7-84a249089008" path="/var/lib/kubelet/pods/7ad49a59-872d-48c0-91f7-84a249089008/volumes"
Oct 30 19:49:31.493922 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:31.493887    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd" podUID="fb610784-5f32-446f-9096-f524917ae3c7" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:49:33.510135 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:33.510098    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8" podUID="8c861815-5c5d-4a6a-9916-fcedf26bc4e0" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:49:41.248261 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:41.248226    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"]
Oct 30 19:49:41.249247 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:41.248523    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq" podUID="969098da-cc93-4414-b2e8-08f2f4ccf4e9" containerName="maas-api" containerID="cri-o://43b7487604a6a0bb69ee51b6c3335febb06a3f44441c1d1d63117a7cc8e7a1aa" gracePeriod=30
Oct 30 19:49:41.345313 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:41.345278    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq" podUID="969098da-cc93-4414-b2e8-08f2f4ccf4e9" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.68:8443/health\": dial tcp 10.133.0.68:8443: connect: connection refused"
Oct 30 19:49:41.494618 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:41.494576    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd" podUID="fb610784-5f32-446f-9096-f524917ae3c7" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:49:41.582992 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:41.582965    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"
Oct 30 19:49:41.760676 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:41.760645    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/969098da-cc93-4414-b2e8-08f2f4ccf4e9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/969098da-cc93-4414-b2e8-08f2f4ccf4e9-maas-api-tls\") pod \"969098da-cc93-4414-b2e8-08f2f4ccf4e9\" (UID: \"969098da-cc93-4414-b2e8-08f2f4ccf4e9\") "
Oct 30 19:49:41.760676 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:41.760681    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/969098da-cc93-4414-b2e8-08f2f4ccf4e9-kube-api-access-hz4v7\" (UniqueName: \"kubernetes.io/projected/969098da-cc93-4414-b2e8-08f2f4ccf4e9-kube-api-access-hz4v7\") pod \"969098da-cc93-4414-b2e8-08f2f4ccf4e9\" (UID: \"969098da-cc93-4414-b2e8-08f2f4ccf4e9\") "
Oct 30 19:49:41.760960 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:41.760726    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/969098da-cc93-4414-b2e8-08f2f4ccf4e9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/969098da-cc93-4414-b2e8-08f2f4ccf4e9-metrics-certs\") pod \"969098da-cc93-4414-b2e8-08f2f4ccf4e9\" (UID: \"969098da-cc93-4414-b2e8-08f2f4ccf4e9\") "
Oct 30 19:49:41.760960 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:41.760842    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/969098da-cc93-4414-b2e8-08f2f4ccf4e9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/969098da-cc93-4414-b2e8-08f2f4ccf4e9-openshift-service-ca\") pod \"969098da-cc93-4414-b2e8-08f2f4ccf4e9\" (UID: \"969098da-cc93-4414-b2e8-08f2f4ccf4e9\") "
Oct 30 19:49:41.761253 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:41.761218    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/969098da-cc93-4414-b2e8-08f2f4ccf4e9-openshift-service-ca" pod "969098da-cc93-4414-b2e8-08f2f4ccf4e9" (UID: "969098da-cc93-4414-b2e8-08f2f4ccf4e9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:41.762845 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:41.762817    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/969098da-cc93-4414-b2e8-08f2f4ccf4e9-maas-api-tls" pod "969098da-cc93-4414-b2e8-08f2f4ccf4e9" (UID: "969098da-cc93-4414-b2e8-08f2f4ccf4e9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:49:41.762953 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:41.762904    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/969098da-cc93-4414-b2e8-08f2f4ccf4e9-metrics-certs" pod "969098da-cc93-4414-b2e8-08f2f4ccf4e9" (UID: "969098da-cc93-4414-b2e8-08f2f4ccf4e9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:49:41.763085 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:41.763067    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/969098da-cc93-4414-b2e8-08f2f4ccf4e9-kube-api-access-hz4v7" pod "969098da-cc93-4414-b2e8-08f2f4ccf4e9" (UID: "969098da-cc93-4414-b2e8-08f2f4ccf4e9"). InnerVolumeSpecName "kube-api-access-hz4v7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:41.862469 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:41.862436    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/969098da-cc93-4414-b2e8-08f2f4ccf4e9-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:41.862469 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:41.862465    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/969098da-cc93-4414-b2e8-08f2f4ccf4e9-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:41.862469 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:41.862475    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hz4v7\" (UniqueName: \"kubernetes.io/projected/969098da-cc93-4414-b2e8-08f2f4ccf4e9-kube-api-access-hz4v7\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:41.862761 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:41.862484    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/969098da-cc93-4414-b2e8-08f2f4ccf4e9-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:42.087436 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:42.087402    4524 generic.go:356] "Generic (PLEG): container finished" podID="969098da-cc93-4414-b2e8-08f2f4ccf4e9" containerID="43b7487604a6a0bb69ee51b6c3335febb06a3f44441c1d1d63117a7cc8e7a1aa" exitCode=0
Oct 30 19:49:42.087643 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:42.087448    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"
Oct 30 19:49:42.087643 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:42.087469    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq" event={"ID":"969098da-cc93-4414-b2e8-08f2f4ccf4e9","Type":"ContainerDied","Data":"43b7487604a6a0bb69ee51b6c3335febb06a3f44441c1d1d63117a7cc8e7a1aa"}
Oct 30 19:49:42.087643 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:42.087511    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq" event={"ID":"969098da-cc93-4414-b2e8-08f2f4ccf4e9","Type":"ContainerDied","Data":"de971166b50d0581bd1a14491ec32dc99e4928488a50ce177a2cc7a25b6507c4"}
Oct 30 19:49:42.087643 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:42.087532    4524 scope.go:122] "RemoveContainer" containerID="43b7487604a6a0bb69ee51b6c3335febb06a3f44441c1d1d63117a7cc8e7a1aa"
Oct 30 19:49:42.098368 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:42.098320    4524 scope.go:122] "RemoveContainer" containerID="43b7487604a6a0bb69ee51b6c3335febb06a3f44441c1d1d63117a7cc8e7a1aa"
Oct 30 19:49:42.098632 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:49:42.098612    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43b7487604a6a0bb69ee51b6c3335febb06a3f44441c1d1d63117a7cc8e7a1aa\": container with ID starting with 43b7487604a6a0bb69ee51b6c3335febb06a3f44441c1d1d63117a7cc8e7a1aa not found: ID does not exist" containerID="43b7487604a6a0bb69ee51b6c3335febb06a3f44441c1d1d63117a7cc8e7a1aa"
Oct 30 19:49:42.098765 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:42.098639    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43b7487604a6a0bb69ee51b6c3335febb06a3f44441c1d1d63117a7cc8e7a1aa"} err="failed to get container status \"43b7487604a6a0bb69ee51b6c3335febb06a3f44441c1d1d63117a7cc8e7a1aa\": rpc error: code = NotFound desc = could not find container \"43b7487604a6a0bb69ee51b6c3335febb06a3f44441c1d1d63117a7cc8e7a1aa\": container with ID starting with 43b7487604a6a0bb69ee51b6c3335febb06a3f44441c1d1d63117a7cc8e7a1aa not found: ID does not exist"
Oct 30 19:49:42.110080 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:42.110061    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"]
Oct 30 19:49:42.114282 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:42.114236    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-17e904-77bcd97b9-9mbrq"]
Oct 30 19:49:42.389932 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:42.389861    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="969098da-cc93-4414-b2e8-08f2f4ccf4e9" path="/var/lib/kubelet/pods/969098da-cc93-4414-b2e8-08f2f4ccf4e9/volumes"
Oct 30 19:49:43.510298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:43.510256    4524 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8" podUID="8c861815-5c5d-4a6a-9916-fcedf26bc4e0" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:49:45.624851 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:49:45.624818    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:49:49.067705 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:49.067661    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"]
Oct 30 19:49:49.068347 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:49.067886    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l" podUID="a764618a-2358-46f2-bd29-a08c4b800a37" containerName="istio-proxy" containerID="cri-o://aebe670b587c00acb61825e57aff17738062ef418d682ef03646d08d34fb8754" gracePeriod=30
Oct 30 19:49:50.495809 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:50.495784    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8"
Oct 30 19:49:50.524857 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:50.524835    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd"
Oct 30 19:49:50.543670 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:50.543644    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb610784-5f32-446f-9096-f524917ae3c7-kube-api-access-79ldv\" (UniqueName: \"kubernetes.io/projected/fb610784-5f32-446f-9096-f524917ae3c7-kube-api-access-79ldv\") pod \"fb610784-5f32-446f-9096-f524917ae3c7\" (UID: \"fb610784-5f32-446f-9096-f524917ae3c7\") "
Oct 30 19:49:50.543894 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:50.543872    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8c861815-5c5d-4a6a-9916-fcedf26bc4e0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c861815-5c5d-4a6a-9916-fcedf26bc4e0-plugins-config-volume\") pod \"8c861815-5c5d-4a6a-9916-fcedf26bc4e0\" (UID: \"8c861815-5c5d-4a6a-9916-fcedf26bc4e0\") "
Oct 30 19:49:50.544082 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:50.544038    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c861815-5c5d-4a6a-9916-fcedf26bc4e0-kube-api-access-5ln5m\" (UniqueName: \"kubernetes.io/projected/8c861815-5c5d-4a6a-9916-fcedf26bc4e0-kube-api-access-5ln5m\") pod \"8c861815-5c5d-4a6a-9916-fcedf26bc4e0\" (UID: \"8c861815-5c5d-4a6a-9916-fcedf26bc4e0\") "
Oct 30 19:49:50.544225 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:50.544095    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb610784-5f32-446f-9096-f524917ae3c7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb610784-5f32-446f-9096-f524917ae3c7-plugins-config-volume\") pod \"fb610784-5f32-446f-9096-f524917ae3c7\" (UID: \"fb610784-5f32-446f-9096-f524917ae3c7\") "
Oct 30 19:49:50.544496 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:50.544450    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c861815-5c5d-4a6a-9916-fcedf26bc4e0-plugins-config-volume" pod "8c861815-5c5d-4a6a-9916-fcedf26bc4e0" (UID: "8c861815-5c5d-4a6a-9916-fcedf26bc4e0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:50.544867 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:50.544843    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb610784-5f32-446f-9096-f524917ae3c7-plugins-config-volume" pod "fb610784-5f32-446f-9096-f524917ae3c7" (UID: "fb610784-5f32-446f-9096-f524917ae3c7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:50.550559 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:50.550536    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c861815-5c5d-4a6a-9916-fcedf26bc4e0-kube-api-access-5ln5m" pod "8c861815-5c5d-4a6a-9916-fcedf26bc4e0" (UID: "8c861815-5c5d-4a6a-9916-fcedf26bc4e0"). InnerVolumeSpecName "kube-api-access-5ln5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:50.552344 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:50.552321    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb610784-5f32-446f-9096-f524917ae3c7-kube-api-access-79ldv" pod "fb610784-5f32-446f-9096-f524917ae3c7" (UID: "fb610784-5f32-446f-9096-f524917ae3c7"). InnerVolumeSpecName "kube-api-access-79ldv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:50.645155 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:50.645120    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79ldv\" (UniqueName: \"kubernetes.io/projected/fb610784-5f32-446f-9096-f524917ae3c7-kube-api-access-79ldv\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:50.645155 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:50.645156    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c861815-5c5d-4a6a-9916-fcedf26bc4e0-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:50.645329 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:50.645182    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ln5m\" (UniqueName: \"kubernetes.io/projected/8c861815-5c5d-4a6a-9916-fcedf26bc4e0-kube-api-access-5ln5m\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:50.645329 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:50.645196    4524 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fb610784-5f32-446f-9096-f524917ae3c7-plugins-config-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:51.145586 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:51.145555    4524 generic.go:356] "Generic (PLEG): container finished" podID="fb610784-5f32-446f-9096-f524917ae3c7" containerID="f780342c2e4527fa0abfbb49e2590ad11a48c7c1a9365ce718fc687df7c5afe2" exitCode=0
Oct 30 19:49:51.145776 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:51.145634    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd"
Oct 30 19:49:51.145776 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:51.145635    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd" event={"ID":"fb610784-5f32-446f-9096-f524917ae3c7","Type":"ContainerDied","Data":"f780342c2e4527fa0abfbb49e2590ad11a48c7c1a9365ce718fc687df7c5afe2"}
Oct 30 19:49:51.145776 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:51.145762    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd" event={"ID":"fb610784-5f32-446f-9096-f524917ae3c7","Type":"ContainerDied","Data":"4c877d6d1a2428beef92c87644835e2eb86eba59ecaa872fffff684dccabbcea"}
Oct 30 19:49:51.145978 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:51.145780    4524 scope.go:122] "RemoveContainer" containerID="f780342c2e4527fa0abfbb49e2590ad11a48c7c1a9365ce718fc687df7c5afe2"
Oct 30 19:49:51.147209 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:51.147161    4524 generic.go:356] "Generic (PLEG): container finished" podID="8c861815-5c5d-4a6a-9916-fcedf26bc4e0" containerID="0e9107cd32a24205c1758795c83c4b03b6a8ef3e9a365f5b5ed6c6af8d1b8158" exitCode=0
Oct 30 19:49:51.147373 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:51.147228    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8"
Oct 30 19:49:51.147373 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:51.147225    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8" event={"ID":"8c861815-5c5d-4a6a-9916-fcedf26bc4e0","Type":"ContainerDied","Data":"0e9107cd32a24205c1758795c83c4b03b6a8ef3e9a365f5b5ed6c6af8d1b8158"}
Oct 30 19:49:51.147373 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:51.147326    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8" event={"ID":"8c861815-5c5d-4a6a-9916-fcedf26bc4e0","Type":"ContainerDied","Data":"96a4ebc5ddb716811883f274d2bcb492a61658efc908917caaf9541b5f5b788f"}
Oct 30 19:49:51.155810 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:51.155793    4524 scope.go:122] "RemoveContainer" containerID="f780342c2e4527fa0abfbb49e2590ad11a48c7c1a9365ce718fc687df7c5afe2"
Oct 30 19:49:51.156092 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:49:51.156073    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f780342c2e4527fa0abfbb49e2590ad11a48c7c1a9365ce718fc687df7c5afe2\": container with ID starting with f780342c2e4527fa0abfbb49e2590ad11a48c7c1a9365ce718fc687df7c5afe2 not found: ID does not exist" containerID="f780342c2e4527fa0abfbb49e2590ad11a48c7c1a9365ce718fc687df7c5afe2"
Oct 30 19:49:51.156203 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:51.156104    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f780342c2e4527fa0abfbb49e2590ad11a48c7c1a9365ce718fc687df7c5afe2"} err="failed to get container status \"f780342c2e4527fa0abfbb49e2590ad11a48c7c1a9365ce718fc687df7c5afe2\": rpc error: code = NotFound desc = could not find container \"f780342c2e4527fa0abfbb49e2590ad11a48c7c1a9365ce718fc687df7c5afe2\": container with ID starting with f780342c2e4527fa0abfbb49e2590ad11a48c7c1a9365ce718fc687df7c5afe2 not found: ID does not exist"
Oct 30 19:49:51.156203 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:51.156126    4524 scope.go:122] "RemoveContainer" containerID="0e9107cd32a24205c1758795c83c4b03b6a8ef3e9a365f5b5ed6c6af8d1b8158"
Oct 30 19:49:51.164692 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:51.164677    4524 scope.go:122] "RemoveContainer" containerID="0e9107cd32a24205c1758795c83c4b03b6a8ef3e9a365f5b5ed6c6af8d1b8158"
Oct 30 19:49:51.164930 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:49:51.164910    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e9107cd32a24205c1758795c83c4b03b6a8ef3e9a365f5b5ed6c6af8d1b8158\": container with ID starting with 0e9107cd32a24205c1758795c83c4b03b6a8ef3e9a365f5b5ed6c6af8d1b8158 not found: ID does not exist" containerID="0e9107cd32a24205c1758795c83c4b03b6a8ef3e9a365f5b5ed6c6af8d1b8158"
Oct 30 19:49:51.164976 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:51.164937    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e9107cd32a24205c1758795c83c4b03b6a8ef3e9a365f5b5ed6c6af8d1b8158"} err="failed to get container status \"0e9107cd32a24205c1758795c83c4b03b6a8ef3e9a365f5b5ed6c6af8d1b8158\": rpc error: code = NotFound desc = could not find container \"0e9107cd32a24205c1758795c83c4b03b6a8ef3e9a365f5b5ed6c6af8d1b8158\": container with ID starting with 0e9107cd32a24205c1758795c83c4b03b6a8ef3e9a365f5b5ed6c6af8d1b8158 not found: ID does not exist"
Oct 30 19:49:51.171201 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:51.171165    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8"]
Oct 30 19:49:51.175560 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:51.175541    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-17e904-c6bd6d9dd-8fhx8"]
Oct 30 19:49:51.187040 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:51.187016    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd"]
Oct 30 19:49:51.193434 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:51.193406    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-17e904-7cdb49b7-v5ztd"]
Oct 30 19:49:51.783296 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:51.783265    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79698dcf56-jbqjp"]
Oct 30 19:49:51.783832 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:51.783514    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-79698dcf56-jbqjp" podUID="5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2" containerName="manager" containerID="cri-o://ccb841f7189b75406d24d1271f0b6d53b5dc026b239af59d4dc3b37ac1fbe04e" gracePeriod=30
Oct 30 19:49:52.138768 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.138744    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79698dcf56-jbqjp"
Oct 30 19:49:52.154460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.154436    4524 generic.go:356] "Generic (PLEG): container finished" podID="5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2" containerID="ccb841f7189b75406d24d1271f0b6d53b5dc026b239af59d4dc3b37ac1fbe04e" exitCode=0
Oct 30 19:49:52.154575 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.154475    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79698dcf56-jbqjp"
Oct 30 19:49:52.154575 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.154520    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79698dcf56-jbqjp" event={"ID":"5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2","Type":"ContainerDied","Data":"ccb841f7189b75406d24d1271f0b6d53b5dc026b239af59d4dc3b37ac1fbe04e"}
Oct 30 19:49:52.154575 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.154569    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79698dcf56-jbqjp" event={"ID":"5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2","Type":"ContainerDied","Data":"522f4c1dfaab8e6bd3694ff5dfd8300b55c30ea736f387b9d502bd67bbeedebf"}
Oct 30 19:49:52.154718 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.154597    4524 scope.go:122] "RemoveContainer" containerID="ccb841f7189b75406d24d1271f0b6d53b5dc026b239af59d4dc3b37ac1fbe04e"
Oct 30 19:49:52.157734 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.157711    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-metrics-certs\") pod \"5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2\" (UID: \"5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2\") "
Oct 30 19:49:52.157879 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.157845    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-kube-api-access-w4gwn\" (UniqueName: \"kubernetes.io/projected/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-kube-api-access-w4gwn\") pod \"5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2\" (UID: \"5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2\") "
Oct 30 19:49:52.157964 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.157902    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-webhook-cert\") pod \"5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2\" (UID: \"5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2\") "
Oct 30 19:49:52.161621 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.161593    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-metrics-certs" pod "5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2" (UID: "5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:49:52.164350 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.164315    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-webhook-cert" pod "5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2" (UID: "5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:49:52.164543 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.164521    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-kube-api-access-w4gwn" pod "5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2" (UID: "5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2"). InnerVolumeSpecName "kube-api-access-w4gwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:52.177302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.177281    4524 scope.go:122] "RemoveContainer" containerID="ccb841f7189b75406d24d1271f0b6d53b5dc026b239af59d4dc3b37ac1fbe04e"
Oct 30 19:49:52.177681 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:49:52.177658    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccb841f7189b75406d24d1271f0b6d53b5dc026b239af59d4dc3b37ac1fbe04e\": container with ID starting with ccb841f7189b75406d24d1271f0b6d53b5dc026b239af59d4dc3b37ac1fbe04e not found: ID does not exist" containerID="ccb841f7189b75406d24d1271f0b6d53b5dc026b239af59d4dc3b37ac1fbe04e"
Oct 30 19:49:52.177776 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.177699    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccb841f7189b75406d24d1271f0b6d53b5dc026b239af59d4dc3b37ac1fbe04e"} err="failed to get container status \"ccb841f7189b75406d24d1271f0b6d53b5dc026b239af59d4dc3b37ac1fbe04e\": rpc error: code = NotFound desc = could not find container \"ccb841f7189b75406d24d1271f0b6d53b5dc026b239af59d4dc3b37ac1fbe04e\": container with ID starting with ccb841f7189b75406d24d1271f0b6d53b5dc026b239af59d4dc3b37ac1fbe04e not found: ID does not exist"
Oct 30 19:49:52.259323 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.259292    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:52.259323 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.259320    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4gwn\" (UniqueName: \"kubernetes.io/projected/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-kube-api-access-w4gwn\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:52.259477 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.259330    4524 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2-webhook-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:52.388413 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.388382    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8c861815-5c5d-4a6a-9916-fcedf26bc4e0" path="/var/lib/kubelet/pods/8c861815-5c5d-4a6a-9916-fcedf26bc4e0/volumes"
Oct 30 19:49:52.388768 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.388755    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb610784-5f32-446f-9096-f524917ae3c7" path="/var/lib/kubelet/pods/fb610784-5f32-446f-9096-f524917ae3c7/volumes"
Oct 30 19:49:52.476529 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.476499    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79698dcf56-jbqjp"]
Oct 30 19:49:52.480387 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:52.480358    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-79698dcf56-jbqjp"]
Oct 30 19:49:53.195997 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.195967    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79698dcf56-bsnzm"]
Oct 30 19:49:53.196683 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196663    4524 cpu_manager.go:414] "containerMap: removing container" podUID="3689eb31-fbba-4cbf-a355-7056ccec100b" containerName="payload-pre-processing"
Oct 30 19:49:53.196772 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196684    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3689eb31-fbba-4cbf-a355-7056ccec100b" containerName="payload-pre-processing"
Oct 30 19:49:53.196772 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196707    4524 cpu_manager.go:414] "containerMap: removing container" podUID="944c40c9-95ff-419a-8429-2e8c9a2eefba" containerName="maas-api"
Oct 30 19:49:53.196772 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196715    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="944c40c9-95ff-419a-8429-2e8c9a2eefba" containerName="maas-api"
Oct 30 19:49:53.196772 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196728    4524 cpu_manager.go:414] "containerMap: removing container" podUID="881f121f-ca0c-4a99-a9d2-d0cca6b54a48" containerName="istio-proxy"
Oct 30 19:49:53.196772 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196734    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="881f121f-ca0c-4a99-a9d2-d0cca6b54a48" containerName="istio-proxy"
Oct 30 19:49:53.196772 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196745    4524 cpu_manager.go:414] "containerMap: removing container" podUID="08eef1b9-f940-48f4-bb6d-b355616ded55" containerName="maas-api"
Oct 30 19:49:53.196772 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196752    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08eef1b9-f940-48f4-bb6d-b355616ded55" containerName="maas-api"
Oct 30 19:49:53.196772 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196767    4524 cpu_manager.go:414] "containerMap: removing container" podUID="5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2" containerName="manager"
Oct 30 19:49:53.196772 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196774    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2" containerName="manager"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196788    4524 cpu_manager.go:414] "containerMap: removing container" podUID="eadb3435-17d0-4041-8210-3b5cfe7d50bc" containerName="maas-api"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196796    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eadb3435-17d0-4041-8210-3b5cfe7d50bc" containerName="maas-api"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196807    4524 cpu_manager.go:414] "containerMap: removing container" podUID="db66fd6a-afe9-47e7-815c-5c66c86081d8" containerName="payload-processing"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196814    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db66fd6a-afe9-47e7-815c-5c66c86081d8" containerName="payload-processing"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196827    4524 cpu_manager.go:414] "containerMap: removing container" podUID="0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196834    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196843    4524 cpu_manager.go:414] "containerMap: removing container" podUID="914487e6-51d5-4f89-9aa7-e6cd7f329a3e" containerName="maas-api"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196850    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="914487e6-51d5-4f89-9aa7-e6cd7f329a3e" containerName="maas-api"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196861    4524 cpu_manager.go:414] "containerMap: removing container" podUID="8b541712-a45c-41a1-bfcc-fc7349f4355b" containerName="maas-api"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196867    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b541712-a45c-41a1-bfcc-fc7349f4355b" containerName="maas-api"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196877    4524 cpu_manager.go:414] "containerMap: removing container" podUID="a03edaaa-eafc-4c25-846a-ff9fab080afc" containerName="maas-api"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196884    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a03edaaa-eafc-4c25-846a-ff9fab080afc" containerName="maas-api"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196901    4524 cpu_manager.go:414] "containerMap: removing container" podUID="1b05d10a-5c87-42d8-8880-2511d27bf716" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196909    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b05d10a-5c87-42d8-8880-2511d27bf716" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196921    4524 cpu_manager.go:414] "containerMap: removing container" podUID="0d49fd92-802a-402c-8098-414925be3659" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196928    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d49fd92-802a-402c-8098-414925be3659" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196943    4524 cpu_manager.go:414] "containerMap: removing container" podUID="e6dd71b4-1741-4da3-99f4-82121e585d89" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196951    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6dd71b4-1741-4da3-99f4-82121e585d89" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196960    4524 cpu_manager.go:414] "containerMap: removing container" podUID="5c074845-90e7-46eb-bed0-9360969cf36e" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196967    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c074845-90e7-46eb-bed0-9360969cf36e" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196977    4524 cpu_manager.go:414] "containerMap: removing container" podUID="1affe672-c981-43ad-bd2c-099648e4be4c" containerName="payload-pre-processing"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196983    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1affe672-c981-43ad-bd2c-099648e4be4c" containerName="payload-pre-processing"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.196994    4524 cpu_manager.go:414] "containerMap: removing container" podUID="b0beff1a-564e-439e-a451-724039ee76a0" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197001    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0beff1a-564e-439e-a451-724039ee76a0" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197012    4524 cpu_manager.go:414] "containerMap: removing container" podUID="7ad49a59-872d-48c0-91f7-84a249089008" containerName="payload-pre-processing"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197020    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ad49a59-872d-48c0-91f7-84a249089008" containerName="payload-pre-processing"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197033    4524 cpu_manager.go:414] "containerMap: removing container" podUID="29e3ef22-7657-4fe5-803e-75a93978aced" containerName="maas-api"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197040    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29e3ef22-7657-4fe5-803e-75a93978aced" containerName="maas-api"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197052    4524 cpu_manager.go:414] "containerMap: removing container" podUID="662686fd-7d04-48b9-bf7e-adbd30299109" containerName="maas-api"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197058    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="662686fd-7d04-48b9-bf7e-adbd30299109" containerName="maas-api"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197071    4524 cpu_manager.go:414] "containerMap: removing container" podUID="edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197078    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197092    4524 cpu_manager.go:414] "containerMap: removing container" podUID="c3f46492-0c06-4e9d-bd74-793bee10103c" containerName="maas-api"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197100    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3f46492-0c06-4e9d-bd74-793bee10103c" containerName="maas-api"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197110    4524 cpu_manager.go:414] "containerMap: removing container" podUID="68c74fd5-f30e-4eb7-bc88-e2b679a28451" containerName="maas-api"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197118    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="68c74fd5-f30e-4eb7-bc88-e2b679a28451" containerName="maas-api"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197128    4524 cpu_manager.go:414] "containerMap: removing container" podUID="5d596f44-3f24-4f49-8baf-6dfe09ad625a" containerName="payload-pre-processing"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197136    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d596f44-3f24-4f49-8baf-6dfe09ad625a" containerName="payload-pre-processing"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197148    4524 cpu_manager.go:414] "containerMap: removing container" podUID="e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0" containerName="main"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197155    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0" containerName="main"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197165    4524 cpu_manager.go:414] "containerMap: removing container" podUID="e53dd061-8aa9-4ace-9140-aea8a7375c20" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197196    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e53dd061-8aa9-4ace-9140-aea8a7375c20" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197207    4524 cpu_manager.go:414] "containerMap: removing container" podUID="1e3ef760-19e1-45bc-b806-57d79f5082db" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197214    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e3ef760-19e1-45bc-b806-57d79f5082db" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197225    4524 cpu_manager.go:414] "containerMap: removing container" podUID="c0210704-8d10-4dad-8f61-7e028f2bbab8" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197232    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c0210704-8d10-4dad-8f61-7e028f2bbab8" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197246    4524 cpu_manager.go:414] "containerMap: removing container" podUID="fb610784-5f32-446f-9096-f524917ae3c7" containerName="payload-pre-processing"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197254    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb610784-5f32-446f-9096-f524917ae3c7" containerName="payload-pre-processing"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197265    4524 cpu_manager.go:414] "containerMap: removing container" podUID="8c861815-5c5d-4a6a-9916-fcedf26bc4e0" containerName="payload-processing"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197272    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c861815-5c5d-4a6a-9916-fcedf26bc4e0" containerName="payload-processing"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197283    4524 cpu_manager.go:414] "containerMap: removing container" podUID="54369e69-c268-48e4-80cd-84021afa2d10" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197290    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54369e69-c268-48e4-80cd-84021afa2d10" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197301    4524 cpu_manager.go:414] "containerMap: removing container" podUID="6fb90c94-7e88-4cd8-96c1-f779013b1694" containerName="maas-api"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197308    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6fb90c94-7e88-4cd8-96c1-f779013b1694" containerName="maas-api"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197318    4524 cpu_manager.go:414] "containerMap: removing container" podUID="caf02c37-523d-43ba-b171-4bcf692ed088" containerName="istio-proxy"
Oct 30 19:49:53.197302 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197325    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="caf02c37-523d-43ba-b171-4bcf692ed088" containerName="istio-proxy"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197338    4524 cpu_manager.go:414] "containerMap: removing container" podUID="969098da-cc93-4414-b2e8-08f2f4ccf4e9" containerName="maas-api"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197345    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="969098da-cc93-4414-b2e8-08f2f4ccf4e9" containerName="maas-api"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197354    4524 cpu_manager.go:414] "containerMap: removing container" podUID="0806e982-fe3c-496d-95e2-c75d2b23dacf" containerName="payload-processing"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197362    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0806e982-fe3c-496d-95e2-c75d2b23dacf" containerName="payload-processing"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197372    4524 cpu_manager.go:414] "containerMap: removing container" podUID="067cde38-0f2e-432e-b278-415e4ce5ff86" containerName="maas-api"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197379    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="067cde38-0f2e-432e-b278-415e4ce5ff86" containerName="maas-api"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197389    4524 cpu_manager.go:414] "containerMap: removing container" podUID="fb3f6371-c4db-4bd9-8101-b34064bb6cc7" containerName="istio-proxy"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197395    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb3f6371-c4db-4bd9-8101-b34064bb6cc7" containerName="istio-proxy"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197409    4524 cpu_manager.go:414] "containerMap: removing container" podUID="7ac2f9eb-a945-483a-9c0a-7ad91b8fd618" containerName="payload-processing"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197416    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ac2f9eb-a945-483a-9c0a-7ad91b8fd618" containerName="payload-processing"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197434    4524 cpu_manager.go:414] "containerMap: removing container" podUID="558458ff-e6e8-41a9-9b1b-c9d0dc6c708e" containerName="maas-api"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197441    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="558458ff-e6e8-41a9-9b1b-c9d0dc6c708e" containerName="maas-api"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197451    4524 cpu_manager.go:414] "containerMap: removing container" podUID="29b2b2ca-a677-4d48-ab5c-9b202e007103" containerName="istio-proxy"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197458    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29b2b2ca-a677-4d48-ab5c-9b202e007103" containerName="istio-proxy"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197560    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="969098da-cc93-4414-b2e8-08f2f4ccf4e9" containerName="maas-api"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197572    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="0806e982-fe3c-496d-95e2-c75d2b23dacf" containerName="payload-processing"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197584    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="067cde38-0f2e-432e-b278-415e4ce5ff86" containerName="maas-api"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197594    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb3f6371-c4db-4bd9-8101-b34064bb6cc7" containerName="istio-proxy"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197605    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ac2f9eb-a945-483a-9c0a-7ad91b8fd618" containerName="payload-processing"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197615    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="558458ff-e6e8-41a9-9b1b-c9d0dc6c708e" containerName="maas-api"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197623    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d49fd92-802a-402c-8098-414925be3659" containerName="istio-proxy"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197632    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="29b2b2ca-a677-4d48-ab5c-9b202e007103" containerName="istio-proxy"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197641    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="3689eb31-fbba-4cbf-a355-7056ccec100b" containerName="payload-pre-processing"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197649    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="944c40c9-95ff-419a-8429-2e8c9a2eefba" containerName="maas-api"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197656    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="881f121f-ca0c-4a99-a9d2-d0cca6b54a48" containerName="istio-proxy"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197666    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="08eef1b9-f940-48f4-bb6d-b355616ded55" containerName="maas-api"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197674    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2" containerName="manager"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197685    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="eadb3435-17d0-4041-8210-3b5cfe7d50bc" containerName="maas-api"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197695    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="db66fd6a-afe9-47e7-815c-5c66c86081d8" containerName="payload-processing"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197705    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="0be49b2f-fdee-4ba1-9e18-3b1edcb6a457" containerName="istio-proxy"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197713    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="914487e6-51d5-4f89-9aa7-e6cd7f329a3e" containerName="maas-api"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197721    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b541712-a45c-41a1-bfcc-fc7349f4355b" containerName="maas-api"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197729    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="a03edaaa-eafc-4c25-846a-ff9fab080afc" containerName="maas-api"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197740    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b05d10a-5c87-42d8-8880-2511d27bf716" containerName="istio-proxy"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197748    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d49fd92-802a-402c-8098-414925be3659" containerName="istio-proxy"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197757    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6dd71b4-1741-4da3-99f4-82121e585d89" containerName="istio-proxy"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197765    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c074845-90e7-46eb-bed0-9360969cf36e" containerName="istio-proxy"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197773    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1affe672-c981-43ad-bd2c-099648e4be4c" containerName="payload-pre-processing"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197782    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="b0beff1a-564e-439e-a451-724039ee76a0" containerName="istio-proxy"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197790    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ad49a59-872d-48c0-91f7-84a249089008" containerName="payload-pre-processing"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197798    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="29e3ef22-7657-4fe5-803e-75a93978aced" containerName="maas-api"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197806    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="662686fd-7d04-48b9-bf7e-adbd30299109" containerName="maas-api"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197815    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="edbb4bd8-4e56-4119-9170-f2e76cdb1eb2" containerName="istio-proxy"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197823    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3f46492-0c06-4e9d-bd74-793bee10103c" containerName="maas-api"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197831    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="68c74fd5-f30e-4eb7-bc88-e2b679a28451" containerName="maas-api"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197840    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d596f44-3f24-4f49-8baf-6dfe09ad625a" containerName="payload-pre-processing"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197849    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="e57b4ffe-99fa-4948-a5d5-f4f0897bd0d0" containerName="main"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197857    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="e53dd061-8aa9-4ace-9140-aea8a7375c20" containerName="istio-proxy"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197865    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e3ef760-19e1-45bc-b806-57d79f5082db" containerName="istio-proxy"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197874    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="c0210704-8d10-4dad-8f61-7e028f2bbab8" containerName="istio-proxy"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197882    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb610784-5f32-446f-9096-f524917ae3c7" containerName="payload-pre-processing"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197890    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c861815-5c5d-4a6a-9916-fcedf26bc4e0" containerName="payload-processing"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197897    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="54369e69-c268-48e4-80cd-84021afa2d10" containerName="istio-proxy"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197905    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="6fb90c94-7e88-4cd8-96c1-f779013b1694" containerName="maas-api"
Oct 30 19:49:53.199786 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.197914    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="caf02c37-523d-43ba-b171-4bcf692ed088" containerName="istio-proxy"
Oct 30 19:49:53.202935 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.202907    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79698dcf56-bsnzm"
Oct 30 19:49:53.211132 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.210872    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 19:49:53.211132 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.210954    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dx5v7\""
Oct 30 19:49:53.211132 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.211049    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 19:49:53.211132 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.211091    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 19:49:53.213853 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.212735    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79698dcf56-bsnzm"]
Oct 30 19:49:53.271227 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.271191    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-metrics-certs\") pod \"maas-controller-79698dcf56-bsnzm\" (UID: \"b1f81ab3-5e93-42fc-837e-bfa36b617fcf\") " pod="opendatahub/maas-controller-79698dcf56-bsnzm"
Oct 30 19:49:53.271403 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.271340    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltrxr\" (UniqueName: \"kubernetes.io/projected/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-kube-api-access-ltrxr\") pod \"maas-controller-79698dcf56-bsnzm\" (UID: \"b1f81ab3-5e93-42fc-837e-bfa36b617fcf\") " pod="opendatahub/maas-controller-79698dcf56-bsnzm"
Oct 30 19:49:53.271403 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.271390    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-webhook-cert\") pod \"maas-controller-79698dcf56-bsnzm\" (UID: \"b1f81ab3-5e93-42fc-837e-bfa36b617fcf\") " pod="opendatahub/maas-controller-79698dcf56-bsnzm"
Oct 30 19:49:53.373206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.372270    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-metrics-certs\") pod \"maas-controller-79698dcf56-bsnzm\" (UID: \"b1f81ab3-5e93-42fc-837e-bfa36b617fcf\") " pod="opendatahub/maas-controller-79698dcf56-bsnzm"
Oct 30 19:49:53.373206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.372365    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltrxr\" (UniqueName: \"kubernetes.io/projected/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-kube-api-access-ltrxr\") pod \"maas-controller-79698dcf56-bsnzm\" (UID: \"b1f81ab3-5e93-42fc-837e-bfa36b617fcf\") " pod="opendatahub/maas-controller-79698dcf56-bsnzm"
Oct 30 19:49:53.373206 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.372403    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-webhook-cert\") pod \"maas-controller-79698dcf56-bsnzm\" (UID: \"b1f81ab3-5e93-42fc-837e-bfa36b617fcf\") " pod="opendatahub/maas-controller-79698dcf56-bsnzm"
Oct 30 19:49:53.375610 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.375540    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-webhook-cert\") pod \"maas-controller-79698dcf56-bsnzm\" (UID: \"b1f81ab3-5e93-42fc-837e-bfa36b617fcf\") " pod="opendatahub/maas-controller-79698dcf56-bsnzm"
Oct 30 19:49:53.375764 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.375661    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-metrics-certs\") pod \"maas-controller-79698dcf56-bsnzm\" (UID: \"b1f81ab3-5e93-42fc-837e-bfa36b617fcf\") " pod="opendatahub/maas-controller-79698dcf56-bsnzm"
Oct 30 19:49:53.384728 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.384700    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltrxr\" (UniqueName: \"kubernetes.io/projected/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-kube-api-access-ltrxr\") pod \"maas-controller-79698dcf56-bsnzm\" (UID: \"b1f81ab3-5e93-42fc-837e-bfa36b617fcf\") " pod="opendatahub/maas-controller-79698dcf56-bsnzm"
Oct 30 19:49:53.530231 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.530107    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79698dcf56-bsnzm"
Oct 30 19:49:53.849318 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.849293    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79698dcf56-bsnzm"]
Oct 30 19:49:53.851458 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:49:53.851427    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1f81ab3_5e93_42fc_837e_bfa36b617fcf.slice/crio-02beb3680dbe802383f7384abb2a7144d7a3481bfd958f3901f022f01aabcdb1 WatchSource:0}: Error finding container 02beb3680dbe802383f7384abb2a7144d7a3481bfd958f3901f022f01aabcdb1: Status 404 returned error can't find the container with id 02beb3680dbe802383f7384abb2a7144d7a3481bfd958f3901f022f01aabcdb1
Oct 30 19:49:53.852901 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:53.852882    4524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:49:54.186972 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.185378    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79698dcf56-bsnzm" event={"ID":"b1f81ab3-5e93-42fc-837e-bfa36b617fcf","Type":"ContainerStarted","Data":"02beb3680dbe802383f7384abb2a7144d7a3481bfd958f3901f022f01aabcdb1"}
Oct 30 19:49:54.357591 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.357566    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:49:54.420192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.419521    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a764618a-2358-46f2-bd29-a08c4b800a37-istio-token\" (UniqueName: \"kubernetes.io/projected/a764618a-2358-46f2-bd29-a08c4b800a37-istio-token\") pod \"a764618a-2358-46f2-bd29-a08c4b800a37\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") "
Oct 30 19:49:54.420192 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.419871    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-credential-socket\") pod \"a764618a-2358-46f2-bd29-a08c4b800a37\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") "
Oct 30 19:49:54.420418 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.420367    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a764618a-2358-46f2-bd29-a08c4b800a37-kube-api-access-fpdj2\" (UniqueName: \"kubernetes.io/projected/a764618a-2358-46f2-bd29-a08c4b800a37-kube-api-access-fpdj2\") pod \"a764618a-2358-46f2-bd29-a08c4b800a37\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") "
Oct 30 19:49:54.435193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.420659    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-istio-envoy\") pod \"a764618a-2358-46f2-bd29-a08c4b800a37\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") "
Oct 30 19:49:54.435193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.420984    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a764618a-2358-46f2-bd29-a08c4b800a37-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a764618a-2358-46f2-bd29-a08c4b800a37-istiod-ca-cert\") pod \"a764618a-2358-46f2-bd29-a08c4b800a37\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") "
Oct 30 19:49:54.435193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.421390    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-workload-certs\") pod \"a764618a-2358-46f2-bd29-a08c4b800a37\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") "
Oct 30 19:49:54.435193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.421713    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-istio-data\") pod \"a764618a-2358-46f2-bd29-a08c4b800a37\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") "
Oct 30 19:49:54.435193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.422047    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a764618a-2358-46f2-bd29-a08c4b800a37-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a764618a-2358-46f2-bd29-a08c4b800a37-istio-podinfo\") pod \"a764618a-2358-46f2-bd29-a08c4b800a37\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") "
Oct 30 19:49:54.435193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.422325    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-workload-socket\") pod \"a764618a-2358-46f2-bd29-a08c4b800a37\" (UID: \"a764618a-2358-46f2-bd29-a08c4b800a37\") "
Oct 30 19:49:54.435193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.433400    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-istio-data" pod "a764618a-2358-46f2-bd29-a08c4b800a37" (UID: "a764618a-2358-46f2-bd29-a08c4b800a37"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:54.435193 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.433591    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-workload-certs" pod "a764618a-2358-46f2-bd29-a08c4b800a37" (UID: "a764618a-2358-46f2-bd29-a08c4b800a37"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:54.436337 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.436077    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a764618a-2358-46f2-bd29-a08c4b800a37-istiod-ca-cert" pod "a764618a-2358-46f2-bd29-a08c4b800a37" (UID: "a764618a-2358-46f2-bd29-a08c4b800a37"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:54.436337 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.436306    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2" path="/var/lib/kubelet/pods/5f1cebbf-5ee2-44a5-9b9e-b5570312c3b2/volumes"
Oct 30 19:49:54.441521 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.439542    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-workload-socket" pod "a764618a-2358-46f2-bd29-a08c4b800a37" (UID: "a764618a-2358-46f2-bd29-a08c4b800a37"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:54.441521 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.439560    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-credential-socket" pod "a764618a-2358-46f2-bd29-a08c4b800a37" (UID: "a764618a-2358-46f2-bd29-a08c4b800a37"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:54.442386 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.442358    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a764618a-2358-46f2-bd29-a08c4b800a37-kube-api-access-fpdj2" pod "a764618a-2358-46f2-bd29-a08c4b800a37" (UID: "a764618a-2358-46f2-bd29-a08c4b800a37"). InnerVolumeSpecName "kube-api-access-fpdj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:54.443039 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.442460    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-istio-envoy" pod "a764618a-2358-46f2-bd29-a08c4b800a37" (UID: "a764618a-2358-46f2-bd29-a08c4b800a37"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:49:54.443039 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.442799    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a764618a-2358-46f2-bd29-a08c4b800a37-istio-podinfo" pod "a764618a-2358-46f2-bd29-a08c4b800a37" (UID: "a764618a-2358-46f2-bd29-a08c4b800a37"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:49:54.443915 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.443887    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a764618a-2358-46f2-bd29-a08c4b800a37-istio-token" pod "a764618a-2358-46f2-bd29-a08c4b800a37" (UID: "a764618a-2358-46f2-bd29-a08c4b800a37"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:54.529658 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.528421    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a764618a-2358-46f2-bd29-a08c4b800a37-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:54.529658 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.528449    4524 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-credential-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:54.529658 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.528465    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpdj2\" (UniqueName: \"kubernetes.io/projected/a764618a-2358-46f2-bd29-a08c4b800a37-kube-api-access-fpdj2\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:54.529658 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.528480    4524 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-istio-envoy\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:54.529658 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.528493    4524 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a764618a-2358-46f2-bd29-a08c4b800a37-istiod-ca-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:54.529658 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.528506    4524 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-workload-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:54.529658 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.528519    4524 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-istio-data\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:54.529658 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.528531    4524 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a764618a-2358-46f2-bd29-a08c4b800a37-istio-podinfo\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:54.529658 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:54.528544    4524 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a764618a-2358-46f2-bd29-a08c4b800a37-workload-socket\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:55.192535 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:55.192504    4524 generic.go:356] "Generic (PLEG): container finished" podID="a764618a-2358-46f2-bd29-a08c4b800a37" containerID="aebe670b587c00acb61825e57aff17738062ef418d682ef03646d08d34fb8754" exitCode=0
Oct 30 19:49:55.192739 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:55.192572    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"
Oct 30 19:49:55.192739 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:55.192608    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l" event={"ID":"a764618a-2358-46f2-bd29-a08c4b800a37","Type":"ContainerDied","Data":"aebe670b587c00acb61825e57aff17738062ef418d682ef03646d08d34fb8754"}
Oct 30 19:49:55.192739 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:55.192685    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l" event={"ID":"a764618a-2358-46f2-bd29-a08c4b800a37","Type":"ContainerDied","Data":"3b5b3f25ffdd2c00f6fbf38e21c519ef0f29605ce61e3ff7d5c59517a213cb30"}
Oct 30 19:49:55.192739 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:55.192719    4524 scope.go:122] "RemoveContainer" containerID="aebe670b587c00acb61825e57aff17738062ef418d682ef03646d08d34fb8754"
Oct 30 19:49:55.194298 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:55.194270    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79698dcf56-bsnzm" event={"ID":"b1f81ab3-5e93-42fc-837e-bfa36b617fcf","Type":"ContainerStarted","Data":"f621148712cf5bc92b4877005367e91f52e87b5e7e2220d38d1215b37cf39cb9"}
Oct 30 19:49:55.194422 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:55.194408    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79698dcf56-bsnzm"
Oct 30 19:49:55.203004 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:55.202981    4524 scope.go:122] "RemoveContainer" containerID="aebe670b587c00acb61825e57aff17738062ef418d682ef03646d08d34fb8754"
Oct 30 19:49:55.203356 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:49:55.203330    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aebe670b587c00acb61825e57aff17738062ef418d682ef03646d08d34fb8754\": container with ID starting with aebe670b587c00acb61825e57aff17738062ef418d682ef03646d08d34fb8754 not found: ID does not exist" containerID="aebe670b587c00acb61825e57aff17738062ef418d682ef03646d08d34fb8754"
Oct 30 19:49:55.203454 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:55.203367    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aebe670b587c00acb61825e57aff17738062ef418d682ef03646d08d34fb8754"} err="failed to get container status \"aebe670b587c00acb61825e57aff17738062ef418d682ef03646d08d34fb8754\": rpc error: code = NotFound desc = could not find container \"aebe670b587c00acb61825e57aff17738062ef418d682ef03646d08d34fb8754\": container with ID starting with aebe670b587c00acb61825e57aff17738062ef418d682ef03646d08d34fb8754 not found: ID does not exist"
Oct 30 19:49:55.220475 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:55.220431    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-79698dcf56-bsnzm" podStartSLOduration=1.6367961640000002 podStartE2EDuration="2.22042004s" totalImagesPullingTime="583.623876ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:49:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:49:55.218904065 +0000 UTC m=+2626.959895142" watchObservedRunningTime="2026-10-30 19:49:55.22042004 +0000 UTC m=+2626.961411116"
Oct 30 19:49:55.234503 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:55.234482    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"]
Oct 30 19:49:55.238604 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:55.238584    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-17e904-openshift-default-7cfdf6f655-zns7l"]
Oct 30 19:49:56.388567 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:49:56.388532    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a764618a-2358-46f2-bd29-a08c4b800a37" path="/var/lib/kubelet/pods/a764618a-2358-46f2-bd29-a08c4b800a37/volumes"
Oct 30 19:50:06.204570 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:06.204539    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79698dcf56-bsnzm"
Oct 30 19:50:11.618182 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.618124    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"]
Oct 30 19:50:11.618585 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.618571    4524 cpu_manager.go:414] "containerMap: removing container" podUID="a764618a-2358-46f2-bd29-a08c4b800a37" containerName="istio-proxy"
Oct 30 19:50:11.618628 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.618586    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a764618a-2358-46f2-bd29-a08c4b800a37" containerName="istio-proxy"
Oct 30 19:50:11.618628 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.618603    4524 cpu_manager.go:414] "containerMap: removing container" podUID="881f121f-ca0c-4a99-a9d2-d0cca6b54a48" containerName="istio-proxy"
Oct 30 19:50:11.618628 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.618607    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="881f121f-ca0c-4a99-a9d2-d0cca6b54a48" containerName="istio-proxy"
Oct 30 19:50:11.618628 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.618622    4524 cpu_manager.go:414] "containerMap: removing container" podUID="0d49fd92-802a-402c-8098-414925be3659" containerName="istio-proxy"
Oct 30 19:50:11.618628 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.618627    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d49fd92-802a-402c-8098-414925be3659" containerName="istio-proxy"
Oct 30 19:50:11.618780 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.618687    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="a764618a-2358-46f2-bd29-a08c4b800a37" containerName="istio-proxy"
Oct 30 19:50:11.618780 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.618699    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="881f121f-ca0c-4a99-a9d2-d0cca6b54a48" containerName="istio-proxy"
Oct 30 19:50:11.623659 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.623638    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.639413 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.639386    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"]
Oct 30 19:50:11.792488 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.792414    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb42d5ce-ed59-413b-9296-d2a0c09f8499-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.792488 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.792450    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb42d5ce-ed59-413b-9296-d2a0c09f8499-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.792488 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.792480    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb42d5ce-ed59-413b-9296-d2a0c09f8499-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.792712 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.792599    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb42d5ce-ed59-413b-9296-d2a0c09f8499-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.792712 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.792666    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb42d5ce-ed59-413b-9296-d2a0c09f8499-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.792712 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.792694    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb42d5ce-ed59-413b-9296-d2a0c09f8499-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.792829 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.792719    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb42d5ce-ed59-413b-9296-d2a0c09f8499-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.792829 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.792763    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2wqb\" (UniqueName: \"kubernetes.io/projected/bb42d5ce-ed59-413b-9296-d2a0c09f8499-kube-api-access-q2wqb\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.792829 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.792814    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb42d5ce-ed59-413b-9296-d2a0c09f8499-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.893440 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.893413    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2wqb\" (UniqueName: \"kubernetes.io/projected/bb42d5ce-ed59-413b-9296-d2a0c09f8499-kube-api-access-q2wqb\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.893583 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.893448    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb42d5ce-ed59-413b-9296-d2a0c09f8499-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.893583 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.893521    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb42d5ce-ed59-413b-9296-d2a0c09f8499-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.893583 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.893550    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb42d5ce-ed59-413b-9296-d2a0c09f8499-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.893731 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.893595    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb42d5ce-ed59-413b-9296-d2a0c09f8499-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.893731 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.893619    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb42d5ce-ed59-413b-9296-d2a0c09f8499-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.893731 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.893654    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb42d5ce-ed59-413b-9296-d2a0c09f8499-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.893731 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.893688    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb42d5ce-ed59-413b-9296-d2a0c09f8499-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.893731 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.893719    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb42d5ce-ed59-413b-9296-d2a0c09f8499-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.894062 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.894042    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb42d5ce-ed59-413b-9296-d2a0c09f8499-workload-certs\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.894124 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.894085    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb42d5ce-ed59-413b-9296-d2a0c09f8499-credential-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.894124 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.894115    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb42d5ce-ed59-413b-9296-d2a0c09f8499-workload-socket\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.894223 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.894207    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb42d5ce-ed59-413b-9296-d2a0c09f8499-istio-data\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.894291 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.894275    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb42d5ce-ed59-413b-9296-d2a0c09f8499-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.895983 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.895957    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb42d5ce-ed59-413b-9296-d2a0c09f8499-istio-envoy\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.896259 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.896241    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb42d5ce-ed59-413b-9296-d2a0c09f8499-istio-podinfo\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.904939 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.904919    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2wqb\" (UniqueName: \"kubernetes.io/projected/bb42d5ce-ed59-413b-9296-d2a0c09f8499-kube-api-access-q2wqb\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.905011 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.904959    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb42d5ce-ed59-413b-9296-d2a0c09f8499-istio-token\") pod \"maas-default-gateway-openshift-default-855f9574cf-6hpnk\" (UID: \"bb42d5ce-ed59-413b-9296-d2a0c09f8499\") " pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:11.940888 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:11.940866    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:12.223925 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:12.223874    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"]
Oct 30 19:50:12.225902 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:50:12.225863    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb42d5ce_ed59_413b_9296_d2a0c09f8499.slice/crio-76e5dac6563c229882df56b94aae3887ebd920cbbfcc922fe6f632b1238192b6 WatchSource:0}: Error finding container 76e5dac6563c229882df56b94aae3887ebd920cbbfcc922fe6f632b1238192b6: Status 404 returned error can't find the container with id 76e5dac6563c229882df56b94aae3887ebd920cbbfcc922fe6f632b1238192b6
Oct 30 19:50:12.228365 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:12.228326    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:50:12.228452 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:12.228414    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:50:12.228505 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:12.228454    4524 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236076Ki","pods":"250"}
Oct 30 19:50:12.297474 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:12.297448    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk" event={"ID":"bb42d5ce-ed59-413b-9296-d2a0c09f8499","Type":"ContainerStarted","Data":"76e5dac6563c229882df56b94aae3887ebd920cbbfcc922fe6f632b1238192b6"}
Oct 30 19:50:13.305410 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:13.305368    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk" event={"ID":"bb42d5ce-ed59-413b-9296-d2a0c09f8499","Type":"ContainerStarted","Data":"23dbaa45e290ec0ee21cc0296571980d9821b5e524bcc7e9277391d8f98b7e71"}
Oct 30 19:50:13.328959 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:13.328913    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk" podStartSLOduration=2.328898708 podStartE2EDuration="2.328898708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:50:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:50:13.327744637 +0000 UTC m=+2645.068735729" watchObservedRunningTime="2026-10-30 19:50:13.328898708 +0000 UTC m=+2645.069889780"
Oct 30 19:50:13.942034 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:13.941994    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:13.943637 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:13.943612    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-6hpnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.190:15021/healthz/ready\": dial tcp 10.133.0.190:15021: connect: connection refused" start-of-body=
Oct 30 19:50:13.943760 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:13.943677    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk" podUID="bb42d5ce-ed59-413b-9296-d2a0c09f8499" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.190:15021/healthz/ready\": dial tcp 10.133.0.190:15021: connect: connection refused"
Oct 30 19:50:14.942222 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:14.942190    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-6hpnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.190:15021/healthz/ready\": dial tcp 10.133.0.190:15021: connect: connection refused" start-of-body=
Oct 30 19:50:14.942696 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:14.942668    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk" podUID="bb42d5ce-ed59-413b-9296-d2a0c09f8499" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.190:15021/healthz/ready\": dial tcp 10.133.0.190:15021: connect: connection refused"
Oct 30 19:50:15.641716 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:50:15.641683    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:50:15.942023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:15.941918    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-6hpnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.190:15021/healthz/ready\": dial tcp 10.133.0.190:15021: connect: connection refused" start-of-body=
Oct 30 19:50:15.942023 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:15.941986    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk" podUID="bb42d5ce-ed59-413b-9296-d2a0c09f8499" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.190:15021/healthz/ready\": dial tcp 10.133.0.190:15021: connect: connection refused"
Oct 30 19:50:16.942122 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:16.942080    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-6hpnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.190:15021/healthz/ready\": dial tcp 10.133.0.190:15021: connect: connection refused" start-of-body=
Oct 30 19:50:16.942328 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:16.942140    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk" podUID="bb42d5ce-ed59-413b-9296-d2a0c09f8499" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.190:15021/healthz/ready\": dial tcp 10.133.0.190:15021: connect: connection refused"
Oct 30 19:50:17.941759 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:17.941719    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-6hpnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.190:15021/healthz/ready\": dial tcp 10.133.0.190:15021: connect: connection refused" start-of-body=
Oct 30 19:50:17.941969 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:17.941781    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk" podUID="bb42d5ce-ed59-413b-9296-d2a0c09f8499" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.190:15021/healthz/ready\": dial tcp 10.133.0.190:15021: connect: connection refused"
Oct 30 19:50:18.941920 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:18.941882    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-6hpnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.190:15021/healthz/ready\": dial tcp 10.133.0.190:15021: connect: connection refused" start-of-body=
Oct 30 19:50:18.942313 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:18.941942    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk" podUID="bb42d5ce-ed59-413b-9296-d2a0c09f8499" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.190:15021/healthz/ready\": dial tcp 10.133.0.190:15021: connect: connection refused"
Oct 30 19:50:20.942460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:20.942369    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-6hpnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.190:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:50:20.942460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:20.942436    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk" podUID="bb42d5ce-ed59-413b-9296-d2a0c09f8499" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.190:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:50:21.983311 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:21.983274    4524 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-855f9574cf-6hpnk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.190:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:50:21.983848 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:21.983325    4524 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk" podUID="bb42d5ce-ed59-413b-9296-d2a0c09f8499" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.190:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:50:22.575146 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:22.575111    4524 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:22.575729 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:22.575693    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:22.576498 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:22.576479    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-855f9574cf-6hpnk"
Oct 30 19:50:25.921980 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:25.921948    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79698dcf56-bsnzm"]
Oct 30 19:50:25.922409 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:25.922149    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-79698dcf56-bsnzm" podUID="b1f81ab3-5e93-42fc-837e-bfa36b617fcf" containerName="manager" containerID="cri-o://f621148712cf5bc92b4877005367e91f52e87b5e7e2220d38d1215b37cf39cb9" gracePeriod=30
Oct 30 19:50:26.189354 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:26.189326    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79698dcf56-bsnzm"
Oct 30 19:50:26.238266 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:26.238237    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-webhook-cert\" (UniqueName: \"kubernetes.io/secret/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-webhook-cert\") pod \"b1f81ab3-5e93-42fc-837e-bfa36b617fcf\" (UID: \"b1f81ab3-5e93-42fc-837e-bfa36b617fcf\") "
Oct 30 19:50:26.238402 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:26.238361    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-kube-api-access-ltrxr\" (UniqueName: \"kubernetes.io/projected/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-kube-api-access-ltrxr\") pod \"b1f81ab3-5e93-42fc-837e-bfa36b617fcf\" (UID: \"b1f81ab3-5e93-42fc-837e-bfa36b617fcf\") "
Oct 30 19:50:26.238462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:26.238398    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-metrics-certs\") pod \"b1f81ab3-5e93-42fc-837e-bfa36b617fcf\" (UID: \"b1f81ab3-5e93-42fc-837e-bfa36b617fcf\") "
Oct 30 19:50:26.240393 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:26.240363    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-webhook-cert" pod "b1f81ab3-5e93-42fc-837e-bfa36b617fcf" (UID: "b1f81ab3-5e93-42fc-837e-bfa36b617fcf"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:50:26.240524 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:26.240490    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-metrics-certs" pod "b1f81ab3-5e93-42fc-837e-bfa36b617fcf" (UID: "b1f81ab3-5e93-42fc-837e-bfa36b617fcf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:50:26.240578 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:26.240543    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-kube-api-access-ltrxr" pod "b1f81ab3-5e93-42fc-837e-bfa36b617fcf" (UID: "b1f81ab3-5e93-42fc-837e-bfa36b617fcf"). InnerVolumeSpecName "kube-api-access-ltrxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:50:26.339592 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:26.339561    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltrxr\" (UniqueName: \"kubernetes.io/projected/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-kube-api-access-ltrxr\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:26.339592 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:26.339590    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:26.339592 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:26.339599    4524 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b1f81ab3-5e93-42fc-837e-bfa36b617fcf-webhook-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:26.380729 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:26.380691    4524 generic.go:356] "Generic (PLEG): container finished" podID="b1f81ab3-5e93-42fc-837e-bfa36b617fcf" containerID="f621148712cf5bc92b4877005367e91f52e87b5e7e2220d38d1215b37cf39cb9" exitCode=0
Oct 30 19:50:26.380856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:26.380736    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79698dcf56-bsnzm" event={"ID":"b1f81ab3-5e93-42fc-837e-bfa36b617fcf","Type":"ContainerDied","Data":"f621148712cf5bc92b4877005367e91f52e87b5e7e2220d38d1215b37cf39cb9"}
Oct 30 19:50:26.380856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:26.380793    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79698dcf56-bsnzm" event={"ID":"b1f81ab3-5e93-42fc-837e-bfa36b617fcf","Type":"ContainerDied","Data":"02beb3680dbe802383f7384abb2a7144d7a3481bfd958f3901f022f01aabcdb1"}
Oct 30 19:50:26.380856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:26.380815    4524 scope.go:122] "RemoveContainer" containerID="f621148712cf5bc92b4877005367e91f52e87b5e7e2220d38d1215b37cf39cb9"
Oct 30 19:50:26.380856 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:26.380743    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79698dcf56-bsnzm"
Oct 30 19:50:26.391053 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:26.391033    4524 scope.go:122] "RemoveContainer" containerID="f621148712cf5bc92b4877005367e91f52e87b5e7e2220d38d1215b37cf39cb9"
Oct 30 19:50:26.391329 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:50:26.391310    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f621148712cf5bc92b4877005367e91f52e87b5e7e2220d38d1215b37cf39cb9\": container with ID starting with f621148712cf5bc92b4877005367e91f52e87b5e7e2220d38d1215b37cf39cb9 not found: ID does not exist" containerID="f621148712cf5bc92b4877005367e91f52e87b5e7e2220d38d1215b37cf39cb9"
Oct 30 19:50:26.391398 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:26.391338    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f621148712cf5bc92b4877005367e91f52e87b5e7e2220d38d1215b37cf39cb9"} err="failed to get container status \"f621148712cf5bc92b4877005367e91f52e87b5e7e2220d38d1215b37cf39cb9\": rpc error: code = NotFound desc = could not find container \"f621148712cf5bc92b4877005367e91f52e87b5e7e2220d38d1215b37cf39cb9\": container with ID starting with f621148712cf5bc92b4877005367e91f52e87b5e7e2220d38d1215b37cf39cb9 not found: ID does not exist"
Oct 30 19:50:26.407953 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:26.407933    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-79698dcf56-bsnzm"]
Oct 30 19:50:26.420704 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:26.420667    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-79698dcf56-bsnzm"]
Oct 30 19:50:27.635154 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:27.635121    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-79698dcf56-d4z2f"]
Oct 30 19:50:27.635589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:27.635532    4524 cpu_manager.go:414] "containerMap: removing container" podUID="b1f81ab3-5e93-42fc-837e-bfa36b617fcf" containerName="manager"
Oct 30 19:50:27.635589 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:27.635543    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1f81ab3-5e93-42fc-837e-bfa36b617fcf" containerName="manager"
Oct 30 19:50:27.635723 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:27.635621    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1f81ab3-5e93-42fc-837e-bfa36b617fcf" containerName="manager"
Oct 30 19:50:27.640048 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:27.640028    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79698dcf56-d4z2f"
Oct 30 19:50:27.643752 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:27.643723    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 19:50:27.643854 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:27.643760    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dx5v7\""
Oct 30 19:50:27.643854 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:27.643786    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 19:50:27.643928 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:27.643870    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 19:50:27.651252 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:27.651230    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79698dcf56-d4z2f"]
Oct 30 19:50:27.752643 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:27.752615    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk5zx\" (UniqueName: \"kubernetes.io/projected/a1569a44-7b5c-42dd-874a-4d014512d27b-kube-api-access-vk5zx\") pod \"maas-controller-79698dcf56-d4z2f\" (UID: \"a1569a44-7b5c-42dd-874a-4d014512d27b\") " pod="opendatahub/maas-controller-79698dcf56-d4z2f"
Oct 30 19:50:27.752798 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:27.752669    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a1569a44-7b5c-42dd-874a-4d014512d27b-webhook-cert\") pod \"maas-controller-79698dcf56-d4z2f\" (UID: \"a1569a44-7b5c-42dd-874a-4d014512d27b\") " pod="opendatahub/maas-controller-79698dcf56-d4z2f"
Oct 30 19:50:27.752798 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:27.752767    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1569a44-7b5c-42dd-874a-4d014512d27b-metrics-certs\") pod \"maas-controller-79698dcf56-d4z2f\" (UID: \"a1569a44-7b5c-42dd-874a-4d014512d27b\") " pod="opendatahub/maas-controller-79698dcf56-d4z2f"
Oct 30 19:50:27.853831 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:27.853802    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a1569a44-7b5c-42dd-874a-4d014512d27b-webhook-cert\") pod \"maas-controller-79698dcf56-d4z2f\" (UID: \"a1569a44-7b5c-42dd-874a-4d014512d27b\") " pod="opendatahub/maas-controller-79698dcf56-d4z2f"
Oct 30 19:50:27.853962 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:27.853848    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1569a44-7b5c-42dd-874a-4d014512d27b-metrics-certs\") pod \"maas-controller-79698dcf56-d4z2f\" (UID: \"a1569a44-7b5c-42dd-874a-4d014512d27b\") " pod="opendatahub/maas-controller-79698dcf56-d4z2f"
Oct 30 19:50:27.853962 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:27.853894    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk5zx\" (UniqueName: \"kubernetes.io/projected/a1569a44-7b5c-42dd-874a-4d014512d27b-kube-api-access-vk5zx\") pod \"maas-controller-79698dcf56-d4z2f\" (UID: \"a1569a44-7b5c-42dd-874a-4d014512d27b\") " pod="opendatahub/maas-controller-79698dcf56-d4z2f"
Oct 30 19:50:27.856274 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:27.856246    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1569a44-7b5c-42dd-874a-4d014512d27b-metrics-certs\") pod \"maas-controller-79698dcf56-d4z2f\" (UID: \"a1569a44-7b5c-42dd-874a-4d014512d27b\") " pod="opendatahub/maas-controller-79698dcf56-d4z2f"
Oct 30 19:50:27.856391 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:27.856320    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a1569a44-7b5c-42dd-874a-4d014512d27b-webhook-cert\") pod \"maas-controller-79698dcf56-d4z2f\" (UID: \"a1569a44-7b5c-42dd-874a-4d014512d27b\") " pod="opendatahub/maas-controller-79698dcf56-d4z2f"
Oct 30 19:50:27.865807 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:27.865783    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk5zx\" (UniqueName: \"kubernetes.io/projected/a1569a44-7b5c-42dd-874a-4d014512d27b-kube-api-access-vk5zx\") pod \"maas-controller-79698dcf56-d4z2f\" (UID: \"a1569a44-7b5c-42dd-874a-4d014512d27b\") " pod="opendatahub/maas-controller-79698dcf56-d4z2f"
Oct 30 19:50:27.974799 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:27.974744    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-79698dcf56-d4z2f"
Oct 30 19:50:28.250982 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:28.250954    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-79698dcf56-d4z2f"]
Oct 30 19:50:28.252067 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:50:28.252042    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1569a44_7b5c_42dd_874a_4d014512d27b.slice/crio-d8b2d3599217215272eca994b69a0fa55c2ed2440eaf21423ed638c942210e72 WatchSource:0}: Error finding container d8b2d3599217215272eca994b69a0fa55c2ed2440eaf21423ed638c942210e72: Status 404 returned error can't find the container with id d8b2d3599217215272eca994b69a0fa55c2ed2440eaf21423ed638c942210e72
Oct 30 19:50:28.389707 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:28.389678    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b1f81ab3-5e93-42fc-837e-bfa36b617fcf" path="/var/lib/kubelet/pods/b1f81ab3-5e93-42fc-837e-bfa36b617fcf/volumes"
Oct 30 19:50:28.393264 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:28.393239    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79698dcf56-d4z2f" event={"ID":"a1569a44-7b5c-42dd-874a-4d014512d27b","Type":"ContainerStarted","Data":"d8b2d3599217215272eca994b69a0fa55c2ed2440eaf21423ed638c942210e72"}
Oct 30 19:50:29.399702 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:29.399664    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-79698dcf56-d4z2f" event={"ID":"a1569a44-7b5c-42dd-874a-4d014512d27b","Type":"ContainerStarted","Data":"3f94f00754578b7ecc1f782169ac9510b21f68563b0235dcded4740c3082f66c"}
Oct 30 19:50:29.400239 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:29.399823    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-79698dcf56-d4z2f"
Oct 30 19:50:29.421660 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:29.421613    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-79698dcf56-d4z2f" podStartSLOduration=1.934068093 podStartE2EDuration="2.421600831s" totalImagesPullingTime="487.532738ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:50:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:50:29.419350237 +0000 UTC m=+2661.160341316" watchObservedRunningTime="2026-10-30 19:50:29.421600831 +0000 UTC m=+2661.162591963"
Oct 30 19:50:40.407424 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:50:40.407391    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-79698dcf56-d4z2f"
Oct 30 19:50:45.624028 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:50:45.623991    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:51:11.433067 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:11.433024    4524 scope.go:122] "RemoveContainer" containerID="a6449277f04a7994ad5176d01956e51e8e005f5174d59d7cdb3c80a3bdc289ae"
Oct 30 19:51:15.623154 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:51:15.623113    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:51:45.623849 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:51:45.623804    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:51:47.583436 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.583395    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"]
Oct 30 19:51:47.583875 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.583608    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj" podUID="80627fa4-301a-4dba-bc98-6fab70d9707c" containerName="discovery" containerID="cri-o://064e8faa05be45264206fbe7235b17b811b4fa586d058196e9fc22e8867e2976" gracePeriod=30
Oct 30 19:51:47.845731 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.845674    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:51:47.913281 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.913251    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/80627fa4-301a-4dba-bc98-6fab70d9707c-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/80627fa4-301a-4dba-bc98-6fab70d9707c-local-certs\") pod \"80627fa4-301a-4dba-bc98-6fab70d9707c\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") "
Oct 30 19:51:47.913460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.913317    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/80627fa4-301a-4dba-bc98-6fab70d9707c-kube-api-access-glq5g\" (UniqueName: \"kubernetes.io/projected/80627fa4-301a-4dba-bc98-6fab70d9707c-kube-api-access-glq5g\") pod \"80627fa4-301a-4dba-bc98-6fab70d9707c\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") "
Oct 30 19:51:47.913460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.913337    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-csr-dns-cert\") pod \"80627fa4-301a-4dba-bc98-6fab70d9707c\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") "
Oct 30 19:51:47.913460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.913366    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-token\" (UniqueName: \"kubernetes.io/projected/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-token\") pod \"80627fa4-301a-4dba-bc98-6fab70d9707c\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") "
Oct 30 19:51:47.913460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.913401    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-csr-ca-configmap\") pod \"80627fa4-301a-4dba-bc98-6fab70d9707c\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") "
Oct 30 19:51:47.913460 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.913434    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-cacerts\" (UniqueName: \"kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-cacerts\") pod \"80627fa4-301a-4dba-bc98-6fab70d9707c\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") "
Oct 30 19:51:47.913752 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.913491    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-kubeconfig\") pod \"80627fa4-301a-4dba-bc98-6fab70d9707c\" (UID: \"80627fa4-301a-4dba-bc98-6fab70d9707c\") "
Oct 30 19:51:47.913941 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.913897    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-csr-ca-configmap" pod "80627fa4-301a-4dba-bc98-6fab70d9707c" (UID: "80627fa4-301a-4dba-bc98-6fab70d9707c"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:51:47.915844 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.915809    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-cacerts" pod "80627fa4-301a-4dba-bc98-6fab70d9707c" (UID: "80627fa4-301a-4dba-bc98-6fab70d9707c"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:51:47.916221 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.916143    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-token" pod "80627fa4-301a-4dba-bc98-6fab70d9707c" (UID: "80627fa4-301a-4dba-bc98-6fab70d9707c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:51:47.916221 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.916156    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80627fa4-301a-4dba-bc98-6fab70d9707c-kube-api-access-glq5g" pod "80627fa4-301a-4dba-bc98-6fab70d9707c" (UID: "80627fa4-301a-4dba-bc98-6fab70d9707c"). InnerVolumeSpecName "kube-api-access-glq5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:51:47.916221 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.916207    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-kubeconfig" pod "80627fa4-301a-4dba-bc98-6fab70d9707c" (UID: "80627fa4-301a-4dba-bc98-6fab70d9707c"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:51:47.916426 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.916250    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-csr-dns-cert" pod "80627fa4-301a-4dba-bc98-6fab70d9707c" (UID: "80627fa4-301a-4dba-bc98-6fab70d9707c"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:51:47.916426 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.916313    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80627fa4-301a-4dba-bc98-6fab70d9707c-local-certs" pod "80627fa4-301a-4dba-bc98-6fab70d9707c" (UID: "80627fa4-301a-4dba-bc98-6fab70d9707c"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:51:47.972419 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.972388    4524 generic.go:356] "Generic (PLEG): container finished" podID="80627fa4-301a-4dba-bc98-6fab70d9707c" containerID="064e8faa05be45264206fbe7235b17b811b4fa586d058196e9fc22e8867e2976" exitCode=0
Oct 30 19:51:47.972578 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.972427    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"
Oct 30 19:51:47.972578 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.972464    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj" event={"ID":"80627fa4-301a-4dba-bc98-6fab70d9707c","Type":"ContainerDied","Data":"064e8faa05be45264206fbe7235b17b811b4fa586d058196e9fc22e8867e2976"}
Oct 30 19:51:47.972578 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.972518    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj" event={"ID":"80627fa4-301a-4dba-bc98-6fab70d9707c","Type":"ContainerDied","Data":"05a741c609695059451ea83e346a11d3eed508b55cfc2bc653df03afc8724146"}
Oct 30 19:51:47.972578 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.972539    4524 scope.go:122] "RemoveContainer" containerID="064e8faa05be45264206fbe7235b17b811b4fa586d058196e9fc22e8867e2976"
Oct 30 19:51:47.989405 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.986936    4524 scope.go:122] "RemoveContainer" containerID="064e8faa05be45264206fbe7235b17b811b4fa586d058196e9fc22e8867e2976"
Oct 30 19:51:47.989737 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:51:47.989715    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"064e8faa05be45264206fbe7235b17b811b4fa586d058196e9fc22e8867e2976\": container with ID starting with 064e8faa05be45264206fbe7235b17b811b4fa586d058196e9fc22e8867e2976 not found: ID does not exist" containerID="064e8faa05be45264206fbe7235b17b811b4fa586d058196e9fc22e8867e2976"
Oct 30 19:51:47.989803 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:47.989747    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"064e8faa05be45264206fbe7235b17b811b4fa586d058196e9fc22e8867e2976"} err="failed to get container status \"064e8faa05be45264206fbe7235b17b811b4fa586d058196e9fc22e8867e2976\": rpc error: code = NotFound desc = could not find container \"064e8faa05be45264206fbe7235b17b811b4fa586d058196e9fc22e8867e2976\": container with ID starting with 064e8faa05be45264206fbe7235b17b811b4fa586d058196e9fc22e8867e2976 not found: ID does not exist"
Oct 30 19:51:48.001663 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:48.001640    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"]
Oct 30 19:51:48.007274 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:48.007251    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b7qjj"]
Oct 30 19:51:48.015114 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:48.015093    4524 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-kubeconfig\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:48.015218 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:48.015115    4524 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/80627fa4-301a-4dba-bc98-6fab70d9707c-local-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:48.015218 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:48.015128    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glq5g\" (UniqueName: \"kubernetes.io/projected/80627fa4-301a-4dba-bc98-6fab70d9707c-kube-api-access-glq5g\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:48.015218 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:48.015136    4524 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-csr-dns-cert\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:48.015218 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:48.015145    4524 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-token\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:48.015218 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:48.015153    4524 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/80627fa4-301a-4dba-bc98-6fab70d9707c-istio-csr-ca-configmap\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:48.015218 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:48.015161    4524 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/80627fa4-301a-4dba-bc98-6fab70d9707c-cacerts\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:51:48.395430 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:51:48.395395    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="80627fa4-301a-4dba-bc98-6fab70d9707c" path="/var/lib/kubelet/pods/80627fa4-301a-4dba-bc98-6fab70d9707c/volumes"
Oct 30 19:52:15.632833 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:52:15.632791    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:52:45.624903 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:52:45.624861    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:53:15.636219 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:53:15.636143    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:53:45.625034 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:53:45.624990    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:54:15.625843 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:54:15.625807    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:54:32.762663 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:32.762617    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"]
Oct 30 19:54:32.763597 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:32.763572    4524 cpu_manager.go:414] "containerMap: removing container" podUID="80627fa4-301a-4dba-bc98-6fab70d9707c" containerName="discovery"
Oct 30 19:54:32.763597 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:32.763595    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="80627fa4-301a-4dba-bc98-6fab70d9707c" containerName="discovery"
Oct 30 19:54:32.763762 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:32.763688    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="80627fa4-301a-4dba-bc98-6fab70d9707c" containerName="discovery"
Oct 30 19:54:32.767059 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:32.767035    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"
Oct 30 19:54:32.779385 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:32.778808    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-a89c71-serving-cert\""
Oct 30 19:54:32.780699 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:32.779439    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"]
Oct 30 19:54:32.832670 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:32.832636    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktgch\" (UniqueName: \"kubernetes.io/projected/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-kube-api-access-ktgch\") pod \"maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b\" (UID: \"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"
Oct 30 19:54:32.832853 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:32.832706    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-metrics-certs\") pod \"maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b\" (UID: \"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"
Oct 30 19:54:32.832907 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:32.832833    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b\" (UID: \"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"
Oct 30 19:54:32.832907 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:32.832891    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b\" (UID: \"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"
Oct 30 19:54:32.933507 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:32.933475    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-metrics-certs\") pod \"maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b\" (UID: \"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"
Oct 30 19:54:32.933507 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:32.933511    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b\" (UID: \"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"
Oct 30 19:54:32.933723 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:32.933532    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b\" (UID: \"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"
Oct 30 19:54:32.933723 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:32.933583    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktgch\" (UniqueName: \"kubernetes.io/projected/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-kube-api-access-ktgch\") pod \"maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b\" (UID: \"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"
Oct 30 19:54:32.934327 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:32.934298    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b\" (UID: \"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"
Oct 30 19:54:32.936141 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:32.936117    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b\" (UID: \"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"
Oct 30 19:54:32.936254 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:32.936233    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-metrics-certs\") pod \"maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b\" (UID: \"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"
Oct 30 19:54:32.953037 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:32.953013    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktgch\" (UniqueName: \"kubernetes.io/projected/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-kube-api-access-ktgch\") pod \"maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b\" (UID: \"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"
Oct 30 19:54:33.085321 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:33.085276    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"
Oct 30 19:54:33.397055 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:33.396955    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"]
Oct 30 19:54:33.403317 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:54:33.403287    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b9f0c69_7fd3_4cae_b8d1_cf67f2766700.slice/crio-6546ed4e7f6c89d2539b66ea66b946f07588bb2ec1d7d98f4653e4c4321e3ebf WatchSource:0}: Error finding container 6546ed4e7f6c89d2539b66ea66b946f07588bb2ec1d7d98f4653e4c4321e3ebf: Status 404 returned error can't find the container with id 6546ed4e7f6c89d2539b66ea66b946f07588bb2ec1d7d98f4653e4c4321e3ebf
Oct 30 19:54:33.991511 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:33.991478    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b" event={"ID":"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700","Type":"ContainerStarted","Data":"05d2736bf2a7834fb6e4ec6ca2866a083ca58e7d4fecdd08b05a1c34a6e5b6b4"}
Oct 30 19:54:33.991946 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:33.991552    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"
Oct 30 19:54:33.991946 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:33.991604    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b" event={"ID":"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700","Type":"ContainerStarted","Data":"6546ed4e7f6c89d2539b66ea66b946f07588bb2ec1d7d98f4653e4c4321e3ebf"}
Oct 30 19:54:34.011354 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:34.011290    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b" podStartSLOduration=1.6218951769999999 podStartE2EDuration="2.011277313s" totalImagesPullingTime="389.382136ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:54:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:54:34.010159703 +0000 UTC m=+2905.751150803" watchObservedRunningTime="2026-10-30 19:54:34.011277313 +0000 UTC m=+2905.752268416"
Oct 30 19:54:40.002542 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:54:40.002512    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"
Oct 30 19:54:45.625311 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:54:45.625270    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:55:15.626353 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:55:15.626314    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:55:25.370530 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:25.370488    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"]
Oct 30 19:55:25.371031 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:25.370762    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b" podUID="7b9f0c69-7fd3-4cae-b8d1-cf67f2766700" containerName="maas-api" containerID="cri-o://05d2736bf2a7834fb6e4ec6ca2866a083ca58e7d4fecdd08b05a1c34a6e5b6b4" gracePeriod=30
Oct 30 19:55:25.629957 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:25.629892    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"
Oct 30 19:55:25.685581 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:25.685352    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-metrics-certs\") pod \"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700\" (UID: \"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700\") "
Oct 30 19:55:25.685581 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:25.685554    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-openshift-service-ca\") pod \"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700\" (UID: \"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700\") "
Oct 30 19:55:25.685832 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:25.685603    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-maas-api-tls\") pod \"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700\" (UID: \"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700\") "
Oct 30 19:55:25.685832 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:25.685635    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-kube-api-access-ktgch\" (UniqueName: \"kubernetes.io/projected/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-kube-api-access-ktgch\") pod \"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700\" (UID: \"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700\") "
Oct 30 19:55:25.686900 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:25.686852    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-openshift-service-ca" pod "7b9f0c69-7fd3-4cae-b8d1-cf67f2766700" (UID: "7b9f0c69-7fd3-4cae-b8d1-cf67f2766700"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:55:25.689203 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:25.689149    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-metrics-certs" pod "7b9f0c69-7fd3-4cae-b8d1-cf67f2766700" (UID: "7b9f0c69-7fd3-4cae-b8d1-cf67f2766700"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:55:25.689355 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:25.689330    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-maas-api-tls" pod "7b9f0c69-7fd3-4cae-b8d1-cf67f2766700" (UID: "7b9f0c69-7fd3-4cae-b8d1-cf67f2766700"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:55:25.697703 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:25.697662    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-kube-api-access-ktgch" pod "7b9f0c69-7fd3-4cae-b8d1-cf67f2766700" (UID: "7b9f0c69-7fd3-4cae-b8d1-cf67f2766700"). InnerVolumeSpecName "kube-api-access-ktgch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:55:25.787341 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:25.787301    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktgch\" (UniqueName: \"kubernetes.io/projected/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-kube-api-access-ktgch\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:25.787341 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:25.787338    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:25.787532 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:25.787353    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:25.787532 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:25.787366    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:26.294707 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:26.294665    4524 generic.go:356] "Generic (PLEG): container finished" podID="7b9f0c69-7fd3-4cae-b8d1-cf67f2766700" containerID="05d2736bf2a7834fb6e4ec6ca2866a083ca58e7d4fecdd08b05a1c34a6e5b6b4" exitCode=0
Oct 30 19:55:26.294919 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:26.294718    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"
Oct 30 19:55:26.294919 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:26.294759    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b" event={"ID":"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700","Type":"ContainerDied","Data":"05d2736bf2a7834fb6e4ec6ca2866a083ca58e7d4fecdd08b05a1c34a6e5b6b4"}
Oct 30 19:55:26.294919 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:26.294832    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b" event={"ID":"7b9f0c69-7fd3-4cae-b8d1-cf67f2766700","Type":"ContainerDied","Data":"6546ed4e7f6c89d2539b66ea66b946f07588bb2ec1d7d98f4653e4c4321e3ebf"}
Oct 30 19:55:26.294919 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:26.294863    4524 scope.go:122] "RemoveContainer" containerID="05d2736bf2a7834fb6e4ec6ca2866a083ca58e7d4fecdd08b05a1c34a6e5b6b4"
Oct 30 19:55:26.305255 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:26.305233    4524 scope.go:122] "RemoveContainer" containerID="05d2736bf2a7834fb6e4ec6ca2866a083ca58e7d4fecdd08b05a1c34a6e5b6b4"
Oct 30 19:55:26.305575 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:55:26.305545    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05d2736bf2a7834fb6e4ec6ca2866a083ca58e7d4fecdd08b05a1c34a6e5b6b4\": container with ID starting with 05d2736bf2a7834fb6e4ec6ca2866a083ca58e7d4fecdd08b05a1c34a6e5b6b4 not found: ID does not exist" containerID="05d2736bf2a7834fb6e4ec6ca2866a083ca58e7d4fecdd08b05a1c34a6e5b6b4"
Oct 30 19:55:26.305669 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:26.305581    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05d2736bf2a7834fb6e4ec6ca2866a083ca58e7d4fecdd08b05a1c34a6e5b6b4"} err="failed to get container status \"05d2736bf2a7834fb6e4ec6ca2866a083ca58e7d4fecdd08b05a1c34a6e5b6b4\": rpc error: code = NotFound desc = could not find container \"05d2736bf2a7834fb6e4ec6ca2866a083ca58e7d4fecdd08b05a1c34a6e5b6b4\": container with ID starting with 05d2736bf2a7834fb6e4ec6ca2866a083ca58e7d4fecdd08b05a1c34a6e5b6b4 not found: ID does not exist"
Oct 30 19:55:26.351940 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:26.351917    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"]
Oct 30 19:55:26.358086 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:26.358064    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-a89c71-64c75f7fcb-b4p4b"]
Oct 30 19:55:26.389607 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:26.389582    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7b9f0c69-7fd3-4cae-b8d1-cf67f2766700" path="/var/lib/kubelet/pods/7b9f0c69-7fd3-4cae-b8d1-cf67f2766700/volumes"
Oct 30 19:55:45.625821 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:55:45.625776    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:55:52.965962 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:52.965926    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q"]
Oct 30 19:55:52.966538 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:52.966186    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q" podUID="2c4e2402-4877-4386-a104-58050b5c0d09" containerName="manager" containerID="cri-o://8dedec230f9544e34cb6d58ff17a7c754cca4413570e78dcf77d38f020c5bceb" gracePeriod=10
Oct 30 19:55:53.209459 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:53.209433    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q"
Oct 30 19:55:53.384985 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:53.384957    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2c4e2402-4877-4386-a104-58050b5c0d09-kube-api-access-mdm66\" (UniqueName: \"kubernetes.io/projected/2c4e2402-4877-4386-a104-58050b5c0d09-kube-api-access-mdm66\") pod \"2c4e2402-4877-4386-a104-58050b5c0d09\" (UID: \"2c4e2402-4877-4386-a104-58050b5c0d09\") "
Oct 30 19:55:53.385221 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:53.385036    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2c4e2402-4877-4386-a104-58050b5c0d09-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c4e2402-4877-4386-a104-58050b5c0d09-extensions-socket-volume\") pod \"2c4e2402-4877-4386-a104-58050b5c0d09\" (UID: \"2c4e2402-4877-4386-a104-58050b5c0d09\") "
Oct 30 19:55:53.385418 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:53.385396    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c4e2402-4877-4386-a104-58050b5c0d09-extensions-socket-volume" pod "2c4e2402-4877-4386-a104-58050b5c0d09" (UID: "2c4e2402-4877-4386-a104-58050b5c0d09"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:53.386919 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:53.386899    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c4e2402-4877-4386-a104-58050b5c0d09-kube-api-access-mdm66" pod "2c4e2402-4877-4386-a104-58050b5c0d09" (UID: "2c4e2402-4877-4386-a104-58050b5c0d09"). InnerVolumeSpecName "kube-api-access-mdm66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:55:53.467932 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:53.467906    4524 generic.go:356] "Generic (PLEG): container finished" podID="2c4e2402-4877-4386-a104-58050b5c0d09" containerID="8dedec230f9544e34cb6d58ff17a7c754cca4413570e78dcf77d38f020c5bceb" exitCode=0
Oct 30 19:55:53.468039 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:53.467946    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q"
Oct 30 19:55:53.468039 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:53.467996    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q" event={"ID":"2c4e2402-4877-4386-a104-58050b5c0d09","Type":"ContainerDied","Data":"8dedec230f9544e34cb6d58ff17a7c754cca4413570e78dcf77d38f020c5bceb"}
Oct 30 19:55:53.468039 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:53.468037    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q" event={"ID":"2c4e2402-4877-4386-a104-58050b5c0d09","Type":"ContainerDied","Data":"9bca1dc2cba469683d1585ed469c201c603e7d80af65777b038da17696be8d0d"}
Oct 30 19:55:53.468149 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:53.468060    4524 scope.go:122] "RemoveContainer" containerID="8dedec230f9544e34cb6d58ff17a7c754cca4413570e78dcf77d38f020c5bceb"
Oct 30 19:55:53.478047 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:53.478033    4524 scope.go:122] "RemoveContainer" containerID="8dedec230f9544e34cb6d58ff17a7c754cca4413570e78dcf77d38f020c5bceb"
Oct 30 19:55:53.478314 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:55:53.478297    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dedec230f9544e34cb6d58ff17a7c754cca4413570e78dcf77d38f020c5bceb\": container with ID starting with 8dedec230f9544e34cb6d58ff17a7c754cca4413570e78dcf77d38f020c5bceb not found: ID does not exist" containerID="8dedec230f9544e34cb6d58ff17a7c754cca4413570e78dcf77d38f020c5bceb"
Oct 30 19:55:53.478372 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:53.478322    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dedec230f9544e34cb6d58ff17a7c754cca4413570e78dcf77d38f020c5bceb"} err="failed to get container status \"8dedec230f9544e34cb6d58ff17a7c754cca4413570e78dcf77d38f020c5bceb\": rpc error: code = NotFound desc = could not find container \"8dedec230f9544e34cb6d58ff17a7c754cca4413570e78dcf77d38f020c5bceb\": container with ID starting with 8dedec230f9544e34cb6d58ff17a7c754cca4413570e78dcf77d38f020c5bceb not found: ID does not exist"
Oct 30 19:55:53.486535 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:53.486518    4524 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c4e2402-4877-4386-a104-58050b5c0d09-extensions-socket-volume\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:53.486603 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:53.486536    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdm66\" (UniqueName: \"kubernetes.io/projected/2c4e2402-4877-4386-a104-58050b5c0d09-kube-api-access-mdm66\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:53.490749 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:53.490731    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q"]
Oct 30 19:55:53.494327 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:53.494309    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-vxz4q"]
Oct 30 19:55:54.388010 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:55:54.387979    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2c4e2402-4877-4386-a104-58050b5c0d09" path="/var/lib/kubelet/pods/2c4e2402-4877-4386-a104-58050b5c0d09/volumes"
Oct 30 19:56:15.624121 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:56:15.624081    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:56:45.624321 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:56:45.624284    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:56:54.012189 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.012137    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-6q7ln"]
Oct 30 19:56:54.012605 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.012556    4524 cpu_manager.go:414] "containerMap: removing container" podUID="2c4e2402-4877-4386-a104-58050b5c0d09" containerName="manager"
Oct 30 19:56:54.012605 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.012567    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2c4e2402-4877-4386-a104-58050b5c0d09" containerName="manager"
Oct 30 19:56:54.012605 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.012590    4524 cpu_manager.go:414] "containerMap: removing container" podUID="7b9f0c69-7fd3-4cae-b8d1-cf67f2766700" containerName="maas-api"
Oct 30 19:56:54.012605 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.012595    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7b9f0c69-7fd3-4cae-b8d1-cf67f2766700" containerName="maas-api"
Oct 30 19:56:54.012769 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.012658    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="2c4e2402-4877-4386-a104-58050b5c0d09" containerName="manager"
Oct 30 19:56:54.012769 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.012671    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="7b9f0c69-7fd3-4cae-b8d1-cf67f2766700" containerName="maas-api"
Oct 30 19:56:54.015832 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.015811    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-6q7ln"
Oct 30 19:56:54.030205 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.030186    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6hscz\""
Oct 30 19:56:54.033333 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.033312    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-6q7ln"]
Oct 30 19:56:54.183511 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.183476    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wzzd\" (UniqueName: \"kubernetes.io/projected/a26d06e0-d4af-40b2-ab67-0e6300a61781-kube-api-access-4wzzd\") pod \"kuadrant-operator-controller-manager-57bb9bfff-6q7ln\" (UID: \"a26d06e0-d4af-40b2-ab67-0e6300a61781\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-6q7ln"
Oct 30 19:56:54.183640 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.183595    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a26d06e0-d4af-40b2-ab67-0e6300a61781-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-6q7ln\" (UID: \"a26d06e0-d4af-40b2-ab67-0e6300a61781\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-6q7ln"
Oct 30 19:56:54.284755 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.284692    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wzzd\" (UniqueName: \"kubernetes.io/projected/a26d06e0-d4af-40b2-ab67-0e6300a61781-kube-api-access-4wzzd\") pod \"kuadrant-operator-controller-manager-57bb9bfff-6q7ln\" (UID: \"a26d06e0-d4af-40b2-ab67-0e6300a61781\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-6q7ln"
Oct 30 19:56:54.284861 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.284786    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a26d06e0-d4af-40b2-ab67-0e6300a61781-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-6q7ln\" (UID: \"a26d06e0-d4af-40b2-ab67-0e6300a61781\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-6q7ln"
Oct 30 19:56:54.285204 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.285161    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a26d06e0-d4af-40b2-ab67-0e6300a61781-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-6q7ln\" (UID: \"a26d06e0-d4af-40b2-ab67-0e6300a61781\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-6q7ln"
Oct 30 19:56:54.296568 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.296551    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wzzd\" (UniqueName: \"kubernetes.io/projected/a26d06e0-d4af-40b2-ab67-0e6300a61781-kube-api-access-4wzzd\") pod \"kuadrant-operator-controller-manager-57bb9bfff-6q7ln\" (UID: \"a26d06e0-d4af-40b2-ab67-0e6300a61781\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-6q7ln"
Oct 30 19:56:54.328445 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.328426    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-6q7ln"
Oct 30 19:56:54.617088 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.617064    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-6q7ln"]
Oct 30 19:56:54.619237 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:56:54.619213    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda26d06e0_d4af_40b2_ab67_0e6300a61781.slice/crio-08171c61935b8d572eef095dc0a3092eb38aa7ae830b2f2a14613dbfa31b101c WatchSource:0}: Error finding container 08171c61935b8d572eef095dc0a3092eb38aa7ae830b2f2a14613dbfa31b101c: Status 404 returned error can't find the container with id 08171c61935b8d572eef095dc0a3092eb38aa7ae830b2f2a14613dbfa31b101c
Oct 30 19:56:54.787125 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.787090    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-6q7ln" event={"ID":"a26d06e0-d4af-40b2-ab67-0e6300a61781","Type":"ContainerStarted","Data":"c2da6f116fb71c09259cf4810126750f1b82ff6892dc580a60025005d9d43bf5"}
Oct 30 19:56:54.787304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.787151    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-6q7ln" event={"ID":"a26d06e0-d4af-40b2-ab67-0e6300a61781","Type":"ContainerStarted","Data":"08171c61935b8d572eef095dc0a3092eb38aa7ae830b2f2a14613dbfa31b101c"}
Oct 30 19:56:54.787304 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.787207    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-6q7ln"
Oct 30 19:56:54.810928 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:56:54.810851    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-6q7ln" podStartSLOduration=1.810836807 podStartE2EDuration="1.810836807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:56:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:56:54.809506937 +0000 UTC m=+3046.550498016" watchObservedRunningTime="2026-10-30 19:56:54.810836807 +0000 UTC m=+3046.551827885"
Oct 30 19:57:05.794472 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:05.794445    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-6q7ln"
Oct 30 19:57:15.690892 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:57:15.690834    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:57:45.626392 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:57:45.626349    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:57:50.426612 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:50.426519    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"]
Oct 30 19:57:50.430934 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:50.430666    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"
Oct 30 19:57:50.441598 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:50.441269    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-5310d3-serving-cert\""
Oct 30 19:57:50.453072 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:50.453040    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"]
Oct 30 19:57:50.539923 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:50.539884    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f68rk\" (UniqueName: \"kubernetes.io/projected/9ea0105f-36e4-43ca-80c3-da354dc31793-kube-api-access-f68rk\") pod \"maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf\" (UID: \"9ea0105f-36e4-43ca-80c3-da354dc31793\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"
Oct 30 19:57:50.540086 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:50.539946    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ea0105f-36e4-43ca-80c3-da354dc31793-metrics-certs\") pod \"maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf\" (UID: \"9ea0105f-36e4-43ca-80c3-da354dc31793\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"
Oct 30 19:57:50.540086 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:50.540013    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ea0105f-36e4-43ca-80c3-da354dc31793-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf\" (UID: \"9ea0105f-36e4-43ca-80c3-da354dc31793\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"
Oct 30 19:57:50.540086 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:50.540050    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ea0105f-36e4-43ca-80c3-da354dc31793-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf\" (UID: \"9ea0105f-36e4-43ca-80c3-da354dc31793\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"
Oct 30 19:57:50.640757 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:50.640725    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f68rk\" (UniqueName: \"kubernetes.io/projected/9ea0105f-36e4-43ca-80c3-da354dc31793-kube-api-access-f68rk\") pod \"maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf\" (UID: \"9ea0105f-36e4-43ca-80c3-da354dc31793\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"
Oct 30 19:57:50.640933 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:50.640768    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ea0105f-36e4-43ca-80c3-da354dc31793-metrics-certs\") pod \"maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf\" (UID: \"9ea0105f-36e4-43ca-80c3-da354dc31793\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"
Oct 30 19:57:50.640933 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:50.640789    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ea0105f-36e4-43ca-80c3-da354dc31793-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf\" (UID: \"9ea0105f-36e4-43ca-80c3-da354dc31793\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"
Oct 30 19:57:50.640933 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:50.640814    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ea0105f-36e4-43ca-80c3-da354dc31793-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf\" (UID: \"9ea0105f-36e4-43ca-80c3-da354dc31793\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"
Oct 30 19:57:50.641476 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:50.641448    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ea0105f-36e4-43ca-80c3-da354dc31793-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf\" (UID: \"9ea0105f-36e4-43ca-80c3-da354dc31793\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"
Oct 30 19:57:50.643337 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:50.643315    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ea0105f-36e4-43ca-80c3-da354dc31793-metrics-certs\") pod \"maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf\" (UID: \"9ea0105f-36e4-43ca-80c3-da354dc31793\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"
Oct 30 19:57:50.643493 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:50.643469    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ea0105f-36e4-43ca-80c3-da354dc31793-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf\" (UID: \"9ea0105f-36e4-43ca-80c3-da354dc31793\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"
Oct 30 19:57:50.653669 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:50.653648    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f68rk\" (UniqueName: \"kubernetes.io/projected/9ea0105f-36e4-43ca-80c3-da354dc31793-kube-api-access-f68rk\") pod \"maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf\" (UID: \"9ea0105f-36e4-43ca-80c3-da354dc31793\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"
Oct 30 19:57:50.754699 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:50.754598    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"
Oct 30 19:57:51.042735 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:51.042599    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"]
Oct 30 19:57:51.045069 ip-10-0-137-200 kubenswrapper[4524]: W1030 19:57:51.045044    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ea0105f_36e4_43ca_80c3_da354dc31793.slice/crio-748c3a56010ffe0b25579b4b523d8729d89f977f25a1ebc21b8d41300426b9c7 WatchSource:0}: Error finding container 748c3a56010ffe0b25579b4b523d8729d89f977f25a1ebc21b8d41300426b9c7: Status 404 returned error can't find the container with id 748c3a56010ffe0b25579b4b523d8729d89f977f25a1ebc21b8d41300426b9c7
Oct 30 19:57:51.046367 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:51.046348    4524 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:57:51.159958 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:51.147883    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf" event={"ID":"9ea0105f-36e4-43ca-80c3-da354dc31793","Type":"ContainerStarted","Data":"748c3a56010ffe0b25579b4b523d8729d89f977f25a1ebc21b8d41300426b9c7"}
Oct 30 19:57:52.158400 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:52.158367    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf" event={"ID":"9ea0105f-36e4-43ca-80c3-da354dc31793","Type":"ContainerStarted","Data":"2e302de695d73b2bdf6d9b3c28580360e1caaac5d828f9ba695ae39eb2f59e42"}
Oct 30 19:57:52.158847 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:52.158541    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"
Oct 30 19:57:52.178955 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:52.178906    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf" podStartSLOduration=1.676996122 podStartE2EDuration="2.178893868s" totalImagesPullingTime="501.897746ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:57:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:57:52.177293013 +0000 UTC m=+3103.918284092" watchObservedRunningTime="2026-10-30 19:57:52.178893868 +0000 UTC m=+3103.919884946"
Oct 30 19:57:58.182862 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:57:58.182831    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"
Oct 30 19:58:15.624913 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:58:15.624876    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:58:44.851656 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:44.851617    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"]
Oct 30 19:58:44.852222 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:44.851919    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf" podUID="9ea0105f-36e4-43ca-80c3-da354dc31793" containerName="maas-api" containerID="cri-o://2e302de695d73b2bdf6d9b3c28580360e1caaac5d828f9ba695ae39eb2f59e42" gracePeriod=30
Oct 30 19:58:45.137965 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.137940    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"
Oct 30 19:58:45.203300 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.203267    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9ea0105f-36e4-43ca-80c3-da354dc31793-kube-api-access-f68rk\" (UniqueName: \"kubernetes.io/projected/9ea0105f-36e4-43ca-80c3-da354dc31793-kube-api-access-f68rk\") pod \"9ea0105f-36e4-43ca-80c3-da354dc31793\" (UID: \"9ea0105f-36e4-43ca-80c3-da354dc31793\") "
Oct 30 19:58:45.203494 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.203324    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9ea0105f-36e4-43ca-80c3-da354dc31793-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ea0105f-36e4-43ca-80c3-da354dc31793-maas-api-tls\") pod \"9ea0105f-36e4-43ca-80c3-da354dc31793\" (UID: \"9ea0105f-36e4-43ca-80c3-da354dc31793\") "
Oct 30 19:58:45.203494 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.203370    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9ea0105f-36e4-43ca-80c3-da354dc31793-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ea0105f-36e4-43ca-80c3-da354dc31793-metrics-certs\") pod \"9ea0105f-36e4-43ca-80c3-da354dc31793\" (UID: \"9ea0105f-36e4-43ca-80c3-da354dc31793\") "
Oct 30 19:58:45.203494 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.203437    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9ea0105f-36e4-43ca-80c3-da354dc31793-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ea0105f-36e4-43ca-80c3-da354dc31793-openshift-service-ca\") pod \"9ea0105f-36e4-43ca-80c3-da354dc31793\" (UID: \"9ea0105f-36e4-43ca-80c3-da354dc31793\") "
Oct 30 19:58:45.203895 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.203856    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ea0105f-36e4-43ca-80c3-da354dc31793-openshift-service-ca" pod "9ea0105f-36e4-43ca-80c3-da354dc31793" (UID: "9ea0105f-36e4-43ca-80c3-da354dc31793"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:58:45.204044 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.204018    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ea0105f-36e4-43ca-80c3-da354dc31793-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:45.205696 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.205672    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ea0105f-36e4-43ca-80c3-da354dc31793-kube-api-access-f68rk" pod "9ea0105f-36e4-43ca-80c3-da354dc31793" (UID: "9ea0105f-36e4-43ca-80c3-da354dc31793"). InnerVolumeSpecName "kube-api-access-f68rk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:45.205936 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.205911    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ea0105f-36e4-43ca-80c3-da354dc31793-maas-api-tls" pod "9ea0105f-36e4-43ca-80c3-da354dc31793" (UID: "9ea0105f-36e4-43ca-80c3-da354dc31793"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:58:45.206008 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.205921    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ea0105f-36e4-43ca-80c3-da354dc31793-metrics-certs" pod "9ea0105f-36e4-43ca-80c3-da354dc31793" (UID: "9ea0105f-36e4-43ca-80c3-da354dc31793"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:58:45.314655 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.304756    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f68rk\" (UniqueName: \"kubernetes.io/projected/9ea0105f-36e4-43ca-80c3-da354dc31793-kube-api-access-f68rk\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:45.314655 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.304802    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ea0105f-36e4-43ca-80c3-da354dc31793-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:45.314655 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.304818    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ea0105f-36e4-43ca-80c3-da354dc31793-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:45.497002 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.496906    4524 generic.go:356] "Generic (PLEG): container finished" podID="9ea0105f-36e4-43ca-80c3-da354dc31793" containerID="2e302de695d73b2bdf6d9b3c28580360e1caaac5d828f9ba695ae39eb2f59e42" exitCode=0
Oct 30 19:58:45.497221 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.497021    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"
Oct 30 19:58:45.497221 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.497050    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf" event={"ID":"9ea0105f-36e4-43ca-80c3-da354dc31793","Type":"ContainerDied","Data":"2e302de695d73b2bdf6d9b3c28580360e1caaac5d828f9ba695ae39eb2f59e42"}
Oct 30 19:58:45.497221 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.497118    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf" event={"ID":"9ea0105f-36e4-43ca-80c3-da354dc31793","Type":"ContainerDied","Data":"748c3a56010ffe0b25579b4b523d8729d89f977f25a1ebc21b8d41300426b9c7"}
Oct 30 19:58:45.497221 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.497150    4524 scope.go:122] "RemoveContainer" containerID="2e302de695d73b2bdf6d9b3c28580360e1caaac5d828f9ba695ae39eb2f59e42"
Oct 30 19:58:45.508361 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.508339    4524 scope.go:122] "RemoveContainer" containerID="2e302de695d73b2bdf6d9b3c28580360e1caaac5d828f9ba695ae39eb2f59e42"
Oct 30 19:58:45.508657 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:58:45.508636    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e302de695d73b2bdf6d9b3c28580360e1caaac5d828f9ba695ae39eb2f59e42\": container with ID starting with 2e302de695d73b2bdf6d9b3c28580360e1caaac5d828f9ba695ae39eb2f59e42 not found: ID does not exist" containerID="2e302de695d73b2bdf6d9b3c28580360e1caaac5d828f9ba695ae39eb2f59e42"
Oct 30 19:58:45.508736 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.508668    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e302de695d73b2bdf6d9b3c28580360e1caaac5d828f9ba695ae39eb2f59e42"} err="failed to get container status \"2e302de695d73b2bdf6d9b3c28580360e1caaac5d828f9ba695ae39eb2f59e42\": rpc error: code = NotFound desc = could not find container \"2e302de695d73b2bdf6d9b3c28580360e1caaac5d828f9ba695ae39eb2f59e42\": container with ID starting with 2e302de695d73b2bdf6d9b3c28580360e1caaac5d828f9ba695ae39eb2f59e42 not found: ID does not exist"
Oct 30 19:58:45.526643 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.526618    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"]
Oct 30 19:58:45.527462 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:45.527445    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-5310d3-7cc49cff97-td2mf"]
Oct 30 19:58:45.626032 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:58:45.625998    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:58:46.390379 ip-10-0-137-200 kubenswrapper[4524]: I1030 19:58:46.390339    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9ea0105f-36e4-43ca-80c3-da354dc31793" path="/var/lib/kubelet/pods/9ea0105f-36e4-43ca-80c3-da354dc31793/volumes"
Oct 30 19:59:15.626364 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:59:15.626319    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 19:59:45.736442 ip-10-0-137-200 kubenswrapper[4524]: E1030 19:59:45.736389    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 20:00:15.624604 ip-10-0-137-200 kubenswrapper[4524]: E1030 20:00:15.624563    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 20:00:18.560488 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:18.560447    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-7dsgm"]
Oct 30 20:00:18.561065 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:18.561048    4524 cpu_manager.go:414] "containerMap: removing container" podUID="9ea0105f-36e4-43ca-80c3-da354dc31793" containerName="maas-api"
Oct 30 20:00:18.561117 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:18.561067    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9ea0105f-36e4-43ca-80c3-da354dc31793" containerName="maas-api"
Oct 30 20:00:18.561210 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:18.561198    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="9ea0105f-36e4-43ca-80c3-da354dc31793" containerName="maas-api"
Oct 30 20:00:18.564850 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:18.564829    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-7dsgm"
Oct 30 20:00:18.578950 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:18.578921    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-7dsgm"]
Oct 30 20:00:18.627159 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:18.627122    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a04469c3-dcd3-49c7-9acd-beccf229b576-maas-api-tls\") pod \"maas-api-595cc648c-7dsgm\" (UID: \"a04469c3-dcd3-49c7-9acd-beccf229b576\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-7dsgm"
Oct 30 20:00:18.627349 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:18.627191    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a04469c3-dcd3-49c7-9acd-beccf229b576-metrics-certs\") pod \"maas-api-595cc648c-7dsgm\" (UID: \"a04469c3-dcd3-49c7-9acd-beccf229b576\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-7dsgm"
Oct 30 20:00:18.627349 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:18.627302    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a04469c3-dcd3-49c7-9acd-beccf229b576-openshift-service-ca\") pod \"maas-api-595cc648c-7dsgm\" (UID: \"a04469c3-dcd3-49c7-9acd-beccf229b576\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-7dsgm"
Oct 30 20:00:18.627349 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:18.627336    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxjh4\" (UniqueName: \"kubernetes.io/projected/a04469c3-dcd3-49c7-9acd-beccf229b576-kube-api-access-nxjh4\") pod \"maas-api-595cc648c-7dsgm\" (UID: \"a04469c3-dcd3-49c7-9acd-beccf229b576\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-7dsgm"
Oct 30 20:00:18.728268 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:18.728234    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a04469c3-dcd3-49c7-9acd-beccf229b576-maas-api-tls\") pod \"maas-api-595cc648c-7dsgm\" (UID: \"a04469c3-dcd3-49c7-9acd-beccf229b576\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-7dsgm"
Oct 30 20:00:18.728500 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:18.728291    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a04469c3-dcd3-49c7-9acd-beccf229b576-metrics-certs\") pod \"maas-api-595cc648c-7dsgm\" (UID: \"a04469c3-dcd3-49c7-9acd-beccf229b576\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-7dsgm"
Oct 30 20:00:18.728500 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:18.728325    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a04469c3-dcd3-49c7-9acd-beccf229b576-openshift-service-ca\") pod \"maas-api-595cc648c-7dsgm\" (UID: \"a04469c3-dcd3-49c7-9acd-beccf229b576\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-7dsgm"
Oct 30 20:00:18.728500 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:18.728343    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxjh4\" (UniqueName: \"kubernetes.io/projected/a04469c3-dcd3-49c7-9acd-beccf229b576-kube-api-access-nxjh4\") pod \"maas-api-595cc648c-7dsgm\" (UID: \"a04469c3-dcd3-49c7-9acd-beccf229b576\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-7dsgm"
Oct 30 20:00:18.729120 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:18.729091    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a04469c3-dcd3-49c7-9acd-beccf229b576-openshift-service-ca\") pod \"maas-api-595cc648c-7dsgm\" (UID: \"a04469c3-dcd3-49c7-9acd-beccf229b576\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-7dsgm"
Oct 30 20:00:18.731493 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:18.731465    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a04469c3-dcd3-49c7-9acd-beccf229b576-metrics-certs\") pod \"maas-api-595cc648c-7dsgm\" (UID: \"a04469c3-dcd3-49c7-9acd-beccf229b576\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-7dsgm"
Oct 30 20:00:18.731493 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:18.731488    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a04469c3-dcd3-49c7-9acd-beccf229b576-maas-api-tls\") pod \"maas-api-595cc648c-7dsgm\" (UID: \"a04469c3-dcd3-49c7-9acd-beccf229b576\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-7dsgm"
Oct 30 20:00:18.740228 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:18.740204    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxjh4\" (UniqueName: \"kubernetes.io/projected/a04469c3-dcd3-49c7-9acd-beccf229b576-kube-api-access-nxjh4\") pod \"maas-api-595cc648c-7dsgm\" (UID: \"a04469c3-dcd3-49c7-9acd-beccf229b576\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-7dsgm"
Oct 30 20:00:18.893249 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:18.893208    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-7dsgm"
Oct 30 20:00:19.189395 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:19.189300    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-7dsgm"]
Oct 30 20:00:19.195562 ip-10-0-137-200 kubenswrapper[4524]: W1030 20:00:19.195525    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda04469c3_dcd3_49c7_9acd_beccf229b576.slice/crio-465d93d451a4f873b9a705b0e0cbf0520c3a33f00fb681fda8200acf4360fc62 WatchSource:0}: Error finding container 465d93d451a4f873b9a705b0e0cbf0520c3a33f00fb681fda8200acf4360fc62: Status 404 returned error can't find the container with id 465d93d451a4f873b9a705b0e0cbf0520c3a33f00fb681fda8200acf4360fc62
Oct 30 20:00:20.173119 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:20.173079    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-7dsgm" event={"ID":"a04469c3-dcd3-49c7-9acd-beccf229b576","Type":"ContainerStarted","Data":"6f5d0ac1860e77f2b5ea140225ff88f40a505be3220fac0cbf4c265eeae109ef"}
Oct 30 20:00:20.173547 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:20.173140    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-7dsgm" event={"ID":"a04469c3-dcd3-49c7-9acd-beccf229b576","Type":"ContainerStarted","Data":"465d93d451a4f873b9a705b0e0cbf0520c3a33f00fb681fda8200acf4360fc62"}
Oct 30 20:00:20.173547 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:20.173278    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-7dsgm"
Oct 30 20:00:20.192786 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:20.192730    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-595cc648c-7dsgm" podStartSLOduration=1.677674458 podStartE2EDuration="2.192715632s" totalImagesPullingTime="515.041174ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:20.191575811 +0000 UTC m=+3251.932566908" watchObservedRunningTime="2026-10-30 20:00:20.192715632 +0000 UTC m=+3251.933706713"
Oct 30 20:00:26.184324 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:26.184292    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-7dsgm"
Oct 30 20:00:26.241606 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:26.241568    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"]
Oct 30 20:00:26.241851 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:26.241805    4524 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt" podUID="502e23a9-33af-4bdc-88fb-21f189d8ec62" containerName="maas-api" containerID="cri-o://9b60e5ebee3861cc5a15047b102d9d3a7f9c7583f0b470c8b26e8d18cd6ef42f" gracePeriod=30
Oct 30 20:00:27.552822 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:27.552783    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"
Oct 30 20:00:27.620474 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:27.620376    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/502e23a9-33af-4bdc-88fb-21f189d8ec62-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/502e23a9-33af-4bdc-88fb-21f189d8ec62-maas-api-tls\") pod \"502e23a9-33af-4bdc-88fb-21f189d8ec62\" (UID: \"502e23a9-33af-4bdc-88fb-21f189d8ec62\") "
Oct 30 20:00:27.620474 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:27.620435    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/502e23a9-33af-4bdc-88fb-21f189d8ec62-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/502e23a9-33af-4bdc-88fb-21f189d8ec62-openshift-service-ca\") pod \"502e23a9-33af-4bdc-88fb-21f189d8ec62\" (UID: \"502e23a9-33af-4bdc-88fb-21f189d8ec62\") "
Oct 30 20:00:27.620729 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:27.620581    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/502e23a9-33af-4bdc-88fb-21f189d8ec62-kube-api-access-8wchf\" (UniqueName: \"kubernetes.io/projected/502e23a9-33af-4bdc-88fb-21f189d8ec62-kube-api-access-8wchf\") pod \"502e23a9-33af-4bdc-88fb-21f189d8ec62\" (UID: \"502e23a9-33af-4bdc-88fb-21f189d8ec62\") "
Oct 30 20:00:27.620729 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:27.620655    4524 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/502e23a9-33af-4bdc-88fb-21f189d8ec62-metrics-certs\" (UniqueName: \"kubernetes.io/secret/502e23a9-33af-4bdc-88fb-21f189d8ec62-metrics-certs\") pod \"502e23a9-33af-4bdc-88fb-21f189d8ec62\" (UID: \"502e23a9-33af-4bdc-88fb-21f189d8ec62\") "
Oct 30 20:00:27.621289 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:27.621229    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/502e23a9-33af-4bdc-88fb-21f189d8ec62-openshift-service-ca" pod "502e23a9-33af-4bdc-88fb-21f189d8ec62" (UID: "502e23a9-33af-4bdc-88fb-21f189d8ec62"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:00:27.622988 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:27.622960    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/502e23a9-33af-4bdc-88fb-21f189d8ec62-kube-api-access-8wchf" pod "502e23a9-33af-4bdc-88fb-21f189d8ec62" (UID: "502e23a9-33af-4bdc-88fb-21f189d8ec62"). InnerVolumeSpecName "kube-api-access-8wchf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:00:27.623445 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:27.623424    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/502e23a9-33af-4bdc-88fb-21f189d8ec62-metrics-certs" pod "502e23a9-33af-4bdc-88fb-21f189d8ec62" (UID: "502e23a9-33af-4bdc-88fb-21f189d8ec62"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:00:27.630783 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:27.630757    4524 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/502e23a9-33af-4bdc-88fb-21f189d8ec62-maas-api-tls" pod "502e23a9-33af-4bdc-88fb-21f189d8ec62" (UID: "502e23a9-33af-4bdc-88fb-21f189d8ec62"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:00:27.721674 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:27.721628    4524 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wchf\" (UniqueName: \"kubernetes.io/projected/502e23a9-33af-4bdc-88fb-21f189d8ec62-kube-api-access-8wchf\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:27.721674 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:27.721673    4524 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/502e23a9-33af-4bdc-88fb-21f189d8ec62-metrics-certs\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:27.721939 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:27.721692    4524 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/502e23a9-33af-4bdc-88fb-21f189d8ec62-maas-api-tls\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:27.721939 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:27.721708    4524 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/502e23a9-33af-4bdc-88fb-21f189d8ec62-openshift-service-ca\") on node \"ip-10-0-137-200.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:28.243226 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:28.243152    4524 generic.go:356] "Generic (PLEG): container finished" podID="502e23a9-33af-4bdc-88fb-21f189d8ec62" containerID="9b60e5ebee3861cc5a15047b102d9d3a7f9c7583f0b470c8b26e8d18cd6ef42f" exitCode=0
Oct 30 20:00:28.243226 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:28.243230    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt" event={"ID":"502e23a9-33af-4bdc-88fb-21f189d8ec62","Type":"ContainerDied","Data":"9b60e5ebee3861cc5a15047b102d9d3a7f9c7583f0b470c8b26e8d18cd6ef42f"}
Oct 30 20:00:28.243528 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:28.243281    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt" event={"ID":"502e23a9-33af-4bdc-88fb-21f189d8ec62","Type":"ContainerDied","Data":"1b52312efca0f32f54abf666c9d71c6e6611e262aeac90590eb612356a836933"}
Oct 30 20:00:28.243528 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:28.243314    4524 scope.go:122] "RemoveContainer" containerID="9b60e5ebee3861cc5a15047b102d9d3a7f9c7583f0b470c8b26e8d18cd6ef42f"
Oct 30 20:00:28.243528 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:28.243473    4524 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"
Oct 30 20:00:28.278192 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:28.278147    4524 scope.go:122] "RemoveContainer" containerID="9b60e5ebee3861cc5a15047b102d9d3a7f9c7583f0b470c8b26e8d18cd6ef42f"
Oct 30 20:00:28.278789 ip-10-0-137-200 kubenswrapper[4524]: E1030 20:00:28.278764    4524 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b60e5ebee3861cc5a15047b102d9d3a7f9c7583f0b470c8b26e8d18cd6ef42f\": container with ID starting with 9b60e5ebee3861cc5a15047b102d9d3a7f9c7583f0b470c8b26e8d18cd6ef42f not found: ID does not exist" containerID="9b60e5ebee3861cc5a15047b102d9d3a7f9c7583f0b470c8b26e8d18cd6ef42f"
Oct 30 20:00:28.278960 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:28.278932    4524 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b60e5ebee3861cc5a15047b102d9d3a7f9c7583f0b470c8b26e8d18cd6ef42f"} err="failed to get container status \"9b60e5ebee3861cc5a15047b102d9d3a7f9c7583f0b470c8b26e8d18cd6ef42f\": rpc error: code = NotFound desc = could not find container \"9b60e5ebee3861cc5a15047b102d9d3a7f9c7583f0b470c8b26e8d18cd6ef42f\": container with ID starting with 9b60e5ebee3861cc5a15047b102d9d3a7f9c7583f0b470c8b26e8d18cd6ef42f not found: ID does not exist"
Oct 30 20:00:28.305280 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:28.305218    4524 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"]
Oct 30 20:00:28.311214 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:28.308964    4524 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-5qltt"]
Oct 30 20:00:28.435193 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:00:28.434292    4524 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="502e23a9-33af-4bdc-88fb-21f189d8ec62" path="/var/lib/kubelet/pods/502e23a9-33af-4bdc-88fb-21f189d8ec62/volumes"
Oct 30 20:00:45.625696 ip-10-0-137-200 kubenswrapper[4524]: E1030 20:00:45.625656    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 20:01:15.625390 ip-10-0-137-200 kubenswrapper[4524]: E1030 20:01:15.625350    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 20:01:24.278598 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:24.278567    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6cf687fb7-fx8th_669db161-f0cc-42e6-b9f6-8cff97eb4891/manager/0.log"
Oct 30 20:01:24.476784 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:24.476754    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-86vsp_67bbe241-581a-468f-a88f-1b48afdfdc1d/manager/0.log"
Oct 30 20:01:25.190425 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:25.190394    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-79698dcf56-d4z2f_a1569a44-7b5c-42dd-874a-4d014512d27b/manager/0.log"
Oct 30 20:01:25.579311 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:25.579287    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-f87fcf56c-kxjpj_fd887584-7460-4fb8-bb36-f826836156c1/manager/0.log"
Oct 30 20:01:25.793817 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:25.793789    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-observability-584f99c788-p45zp_bca23000-fcb5-41f7-9051-f5feaa096fe4/manager/0.log"
Oct 30 20:01:26.530703 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:26.530676    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c94c66984-zfkdh_8b72130d-260a-4550-a9b6-3caeedcedb71/manager/0.log"
Oct 30 20:01:26.550735 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:26.550711    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c94c66984-zfkdh_8b72130d-260a-4550-a9b6-3caeedcedb71/copy-manifests/0.log"
Oct 30 20:01:26.741363 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:26.741335    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-7d94f55f4b-zsd4c_7be79571-1994-41af-a1e7-63ce281f44a2/proxy/0.log"
Oct 30 20:01:27.522305 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:27.522276    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-7dsgm_a04469c3-dcd3-49c7-9acd-beccf229b576/maas-api/0.log"
Oct 30 20:01:29.549939 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:29.549906    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-wv55z_5d311a0e-284d-4ae2-b37d-0c0542b916fc/postgres/0.log"
Oct 30 20:01:31.513904 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:31.513873    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-v9g7j_201b4047-ff95-46c1-a3e2-c26c90093f6b/manager/0.log"
Oct 30 20:01:31.907975 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:31.907939    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57bb9bfff-6q7ln_a26d06e0-d4af-40b2-ab67-0e6300a61781/manager/0.log"
Oct 30 20:01:33.297749 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:33.297709    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-2ltm4_741a2f85-9f59-443e-909f-ff76c3816645/discovery/0.log"
Oct 30 20:01:33.667389 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:33.667360    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-678bf485d7-kpxzm_b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2/kube-auth-proxy/0.log"
Oct 30 20:01:33.856769 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:33.856740    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-6hpnk_bb42d5ce-ed59-413b-9296-d2a0c09f8499/istio-proxy/0.log"
Oct 30 20:01:34.037228 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:34.037111    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-bt7bg_cb7c8396-fff1-47c2-9def-f721fb1eaeb4/istio-proxy/0.log"
Oct 30 20:01:34.988263 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:34.988232    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-rc4rt_64b8f203-6e98-4ce3-adea-8686640b74e8/istio-proxy/0.log"
Oct 30 20:01:37.094727 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:37.094693    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n_7f53bd27-b284-4538-8b5e-24a32917541b/main/0.log"
Oct 30 20:01:37.456109 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:37.456027    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-zgk94_d44ab249-0b8f-4f57-ae7f-b6071a229e01/main/0.log"
Oct 30 20:01:37.640843 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:37.640817    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-vc5q8_28cfff8e-626d-4a42-8d25-b5296c17db9c/main/0.log"
Oct 30 20:01:37.822012 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:37.821941    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79_420a5dfa-7b17-480e-bbb7-71f7c290f24e/main/0.log"
Oct 30 20:01:45.624544 ip-10-0-137-200 kubenswrapper[4524]: E1030 20:01:45.624504    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 20:01:48.383458 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:48.383418    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6cf687fb7-fx8th_669db161-f0cc-42e6-b9f6-8cff97eb4891/manager/0.log"
Oct 30 20:01:48.592874 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:48.592823    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-86vsp_67bbe241-581a-468f-a88f-1b48afdfdc1d/manager/0.log"
Oct 30 20:01:49.229914 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:49.229881    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-79698dcf56-d4z2f_a1569a44-7b5c-42dd-874a-4d014512d27b/manager/0.log"
Oct 30 20:01:49.631491 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:49.631456    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-f87fcf56c-kxjpj_fd887584-7460-4fb8-bb36-f826836156c1/manager/0.log"
Oct 30 20:01:49.851645 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:49.851613    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-observability-584f99c788-p45zp_bca23000-fcb5-41f7-9051-f5feaa096fe4/manager/0.log"
Oct 30 20:01:50.572132 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:50.572100    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c94c66984-zfkdh_8b72130d-260a-4550-a9b6-3caeedcedb71/copy-manifests/0.log"
Oct 30 20:01:50.596756 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:50.596732    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c94c66984-zfkdh_8b72130d-260a-4550-a9b6-3caeedcedb71/manager/0.log"
Oct 30 20:01:50.783960 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:50.783930    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-7d94f55f4b-zsd4c_7be79571-1994-41af-a1e7-63ce281f44a2/proxy/0.log"
Oct 30 20:01:51.343618 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:51.343587    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-7dsgm_a04469c3-dcd3-49c7-9acd-beccf229b576/maas-api/0.log"
Oct 30 20:01:53.376714 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:53.376687    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-wv55z_5d311a0e-284d-4ae2-b37d-0c0542b916fc/postgres/0.log"
Oct 30 20:01:54.933708 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:54.933677    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-v9g7j_201b4047-ff95-46c1-a3e2-c26c90093f6b/manager/0.log"
Oct 30 20:01:55.323868 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:55.323788    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57bb9bfff-6q7ln_a26d06e0-d4af-40b2-ab67-0e6300a61781/manager/0.log"
Oct 30 20:01:56.507057 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:56.507024    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-2ltm4_741a2f85-9f59-443e-909f-ff76c3816645/discovery/0.log"
Oct 30 20:01:56.878895 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:56.878869    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-678bf485d7-kpxzm_b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2/kube-auth-proxy/0.log"
Oct 30 20:01:57.072923 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:57.072891    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-6hpnk_bb42d5ce-ed59-413b-9296-d2a0c09f8499/istio-proxy/0.log"
Oct 30 20:01:57.247707 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:57.247633    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-bt7bg_cb7c8396-fff1-47c2-9def-f721fb1eaeb4/istio-proxy/0.log"
Oct 30 20:01:58.217285 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:58.217249    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-rc4rt_64b8f203-6e98-4ce3-adea-8686640b74e8/istio-proxy/0.log"
Oct 30 20:01:59.853025 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:01:59.853004    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-qn67n_7f53bd27-b284-4538-8b5e-24a32917541b/main/0.log"
Oct 30 20:02:00.211241 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:02:00.211145    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-zgk94_d44ab249-0b8f-4f57-ae7f-b6071a229e01/main/0.log"
Oct 30 20:02:00.383020 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:02:00.382992    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-vc5q8_28cfff8e-626d-4a42-8d25-b5296c17db9c/main/0.log"
Oct 30 20:02:00.577275 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:02:00.577189    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8wbc79_420a5dfa-7b17-480e-bbb7-71f7c290f24e/main/0.log"
Oct 30 20:02:15.624075 ip-10-0-137-200 kubenswrapper[4524]: E1030 20:02:15.624034    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 20:02:38.153363 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:02:38.153331    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-79698dcf56-d4z2f_a1569a44-7b5c-42dd-874a-4d014512d27b/manager/0.log"
Oct 30 20:02:38.441724 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:02:38.441653    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-79698dcf56-d4z2f_a1569a44-7b5c-42dd-874a-4d014512d27b/manager/0.log"
Oct 30 20:02:45.624246 ip-10-0-137-200 kubenswrapper[4524]: E1030 20:02:45.624209    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 20:03:15.624418 ip-10-0-137-200 kubenswrapper[4524]: E1030 20:03:15.624380    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 20:03:23.769953 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:23.769917    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-79698dcf56-d4z2f_a1569a44-7b5c-42dd-874a-4d014512d27b/manager/0.log"
Oct 30 20:03:24.055877 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:24.055786    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-79698dcf56-d4z2f_a1569a44-7b5c-42dd-874a-4d014512d27b/manager/0.log"
Oct 30 20:03:28.859556 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:28.859510    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7q5vl_ba82c577-b89d-44f7-87e4-2d00d1e2a5de/global-pull-secret-syncer/0.log"
Oct 30 20:03:29.310081 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:29.309974    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ln5ql_8d23c4bb-0012-4e09-bc59-c6689f9ea394/konnectivity-agent/0.log"
Oct 30 20:03:29.491918 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:29.491886    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-200.us-east-2.compute.internal_7e14d55fcbb7a4643b4d48f7935b8114/haproxy/0.log"
Oct 30 20:03:36.525137 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:36.525106    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-v9g7j_201b4047-ff95-46c1-a3e2-c26c90093f6b/manager/0.log"
Oct 30 20:03:36.632400 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:36.632360    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57bb9bfff-6q7ln_a26d06e0-d4af-40b2-ab67-0e6300a61781/manager/0.log"
Oct 30 20:03:38.686945 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:38.686907    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-88zd5_caf4897d-8696-4083-b8d5-77714c6162e9/node-exporter/0.log"
Oct 30 20:03:38.717339 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:38.717313    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-88zd5_caf4897d-8696-4083-b8d5-77714c6162e9/kube-rbac-proxy/0.log"
Oct 30 20:03:38.745915 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:38.745892    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-88zd5_caf4897d-8696-4083-b8d5-77714c6162e9/init-textfile/0.log"
Oct 30 20:03:39.263755 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.263719    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-krs4x_ef933f88-83c6-4616-80de-e30974037645/prometheus-operator/0.log"
Oct 30 20:03:39.287515 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.287480    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-krs4x_ef933f88-83c6-4616-80de-e30974037645/kube-rbac-proxy/0.log"
Oct 30 20:03:39.324587 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.324561    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-ztp7b_8f4e6b19-4c02-43e5-ac9b-fb80af56e5f0/prometheus-operator-admission-webhook/0.log"
Oct 30 20:03:39.365810 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.365781    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5bbccbcb8-p4h88_10e0b1be-f568-4df2-a87a-68468d34835b/telemeter-client/0.log"
Oct 30 20:03:39.395098 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.395070    4524 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"]
Oct 30 20:03:39.395625 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.395605    4524 cpu_manager.go:414] "containerMap: removing container" podUID="502e23a9-33af-4bdc-88fb-21f189d8ec62" containerName="maas-api"
Oct 30 20:03:39.395704 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.395630    4524 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="502e23a9-33af-4bdc-88fb-21f189d8ec62" containerName="maas-api"
Oct 30 20:03:39.395786 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.395774    4524 memory_manager.go:365] "RemoveStaleState removing state" podUID="502e23a9-33af-4bdc-88fb-21f189d8ec62" containerName="maas-api"
Oct 30 20:03:39.398117 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.398097    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"
Oct 30 20:03:39.412798 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.412777    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"]
Oct 30 20:03:39.413021 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.412994    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5bbccbcb8-p4h88_10e0b1be-f568-4df2-a87a-68468d34835b/reload/0.log"
Oct 30 20:03:39.450214 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.450189    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5bbccbcb8-p4h88_10e0b1be-f568-4df2-a87a-68468d34835b/kube-rbac-proxy/0.log"
Oct 30 20:03:39.451424 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.451400    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhlzl\" (UniqueName: \"kubernetes.io/projected/f143ba4e-b0e4-4a77-85ac-6e68d865f298-kube-api-access-bhlzl\") pod \"perf-node-gather-daemonset-wzfm7\" (UID: \"f143ba4e-b0e4-4a77-85ac-6e68d865f298\") " pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"
Oct 30 20:03:39.451641 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.451623    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f143ba4e-b0e4-4a77-85ac-6e68d865f298-podres\") pod \"perf-node-gather-daemonset-wzfm7\" (UID: \"f143ba4e-b0e4-4a77-85ac-6e68d865f298\") " pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"
Oct 30 20:03:39.451753 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.451650    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f143ba4e-b0e4-4a77-85ac-6e68d865f298-sys\") pod \"perf-node-gather-daemonset-wzfm7\" (UID: \"f143ba4e-b0e4-4a77-85ac-6e68d865f298\") " pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"
Oct 30 20:03:39.451753 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.451737    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f143ba4e-b0e4-4a77-85ac-6e68d865f298-lib-modules\") pod \"perf-node-gather-daemonset-wzfm7\" (UID: \"f143ba4e-b0e4-4a77-85ac-6e68d865f298\") " pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"
Oct 30 20:03:39.451867 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.451851    4524 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f143ba4e-b0e4-4a77-85ac-6e68d865f298-proc\") pod \"perf-node-gather-daemonset-wzfm7\" (UID: \"f143ba4e-b0e4-4a77-85ac-6e68d865f298\") " pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"
Oct 30 20:03:39.493998 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.493970    4524 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-trmds\"/\"default-dockercfg-mnw2j\""
Oct 30 20:03:39.493998 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.493975    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-trmds\"/\"kube-root-ca.crt\""
Oct 30 20:03:39.493998 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.493999    4524 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-trmds\"/\"openshift-service-ca.crt\""
Oct 30 20:03:39.552790 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.552719    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f143ba4e-b0e4-4a77-85ac-6e68d865f298-podres\") pod \"perf-node-gather-daemonset-wzfm7\" (UID: \"f143ba4e-b0e4-4a77-85ac-6e68d865f298\") " pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"
Oct 30 20:03:39.552790 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.552749    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f143ba4e-b0e4-4a77-85ac-6e68d865f298-sys\") pod \"perf-node-gather-daemonset-wzfm7\" (UID: \"f143ba4e-b0e4-4a77-85ac-6e68d865f298\") " pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"
Oct 30 20:03:39.552790 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.552782    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f143ba4e-b0e4-4a77-85ac-6e68d865f298-lib-modules\") pod \"perf-node-gather-daemonset-wzfm7\" (UID: \"f143ba4e-b0e4-4a77-85ac-6e68d865f298\") " pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"
Oct 30 20:03:39.552987 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.552815    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f143ba4e-b0e4-4a77-85ac-6e68d865f298-proc\") pod \"perf-node-gather-daemonset-wzfm7\" (UID: \"f143ba4e-b0e4-4a77-85ac-6e68d865f298\") " pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"
Oct 30 20:03:39.552987 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.552849    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f143ba4e-b0e4-4a77-85ac-6e68d865f298-podres\") pod \"perf-node-gather-daemonset-wzfm7\" (UID: \"f143ba4e-b0e4-4a77-85ac-6e68d865f298\") " pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"
Oct 30 20:03:39.552987 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.552862    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f143ba4e-b0e4-4a77-85ac-6e68d865f298-sys\") pod \"perf-node-gather-daemonset-wzfm7\" (UID: \"f143ba4e-b0e4-4a77-85ac-6e68d865f298\") " pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"
Oct 30 20:03:39.552987 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.552852    4524 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhlzl\" (UniqueName: \"kubernetes.io/projected/f143ba4e-b0e4-4a77-85ac-6e68d865f298-kube-api-access-bhlzl\") pod \"perf-node-gather-daemonset-wzfm7\" (UID: \"f143ba4e-b0e4-4a77-85ac-6e68d865f298\") " pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"
Oct 30 20:03:39.552987 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.552934    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f143ba4e-b0e4-4a77-85ac-6e68d865f298-proc\") pod \"perf-node-gather-daemonset-wzfm7\" (UID: \"f143ba4e-b0e4-4a77-85ac-6e68d865f298\") " pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"
Oct 30 20:03:39.553204 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.553010    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f143ba4e-b0e4-4a77-85ac-6e68d865f298-lib-modules\") pod \"perf-node-gather-daemonset-wzfm7\" (UID: \"f143ba4e-b0e4-4a77-85ac-6e68d865f298\") " pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"
Oct 30 20:03:39.564569 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.564549    4524 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhlzl\" (UniqueName: \"kubernetes.io/projected/f143ba4e-b0e4-4a77-85ac-6e68d865f298-kube-api-access-bhlzl\") pod \"perf-node-gather-daemonset-wzfm7\" (UID: \"f143ba4e-b0e4-4a77-85ac-6e68d865f298\") " pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"
Oct 30 20:03:39.709815 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.709779    4524 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"
Oct 30 20:03:39.988401 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:39.988376    4524 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"]
Oct 30 20:03:39.990056 ip-10-0-137-200 kubenswrapper[4524]: W1030 20:03:39.990032    4524 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf143ba4e_b0e4_4a77_85ac_6e68d865f298.slice/crio-2a7ae3ba74eb42d1032c6e61439f599700e6479c05bb237f85eee8d0b4d8c057 WatchSource:0}: Error finding container 2a7ae3ba74eb42d1032c6e61439f599700e6479c05bb237f85eee8d0b4d8c057: Status 404 returned error can't find the container with id 2a7ae3ba74eb42d1032c6e61439f599700e6479c05bb237f85eee8d0b4d8c057
Oct 30 20:03:40.335974 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:40.335941    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7" event={"ID":"f143ba4e-b0e4-4a77-85ac-6e68d865f298","Type":"ContainerStarted","Data":"d3990b09a63bd284d016b776456f33a708a49da220970639f66ce050ea12cc87"}
Oct 30 20:03:40.336148 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:40.335993    4524 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7" event={"ID":"f143ba4e-b0e4-4a77-85ac-6e68d865f298","Type":"ContainerStarted","Data":"2a7ae3ba74eb42d1032c6e61439f599700e6479c05bb237f85eee8d0b4d8c057"}
Oct 30 20:03:40.336148 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:40.336070    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"
Oct 30 20:03:40.360554 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:40.360510    4524 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7" podStartSLOduration=1.3604975449999999 podStartE2EDuration="1.360497545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:03:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:03:40.35742219 +0000 UTC m=+3452.098413270" watchObservedRunningTime="2026-10-30 20:03:40.360497545 +0000 UTC m=+3452.101488624"
Oct 30 20:03:42.101698 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:42.101668    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6dd789c8b7-v6lqx_5e25a61d-8a82-40ba-aad3-81ad9f762df7/download-server/0.log"
Oct 30 20:03:44.044829 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:44.044794    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v86zw_ce07e9b3-5e98-4d16-8158-afba6fe42281/dns/0.log"
Oct 30 20:03:44.073851 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:44.073824    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-v86zw_ce07e9b3-5e98-4d16-8158-afba6fe42281/kube-rbac-proxy/0.log"
Oct 30 20:03:44.208046 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:44.208018    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8jrs8_94709da2-704a-421c-aa2f-1844db5bec96/dns-node-resolver/0.log"
Oct 30 20:03:44.894618 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:44.894583    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6b4449ff4b-jmtvm_9f98e875-33c4-473e-889b-223453fdd51b/registry/0.log"
Oct 30 20:03:44.926332 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:44.926298    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2z2fd_838f8f5e-ac73-4280-8ed3-173a51c42b9e/node-ca/0.log"
Oct 30 20:03:45.624982 ip-10-0-137-200 kubenswrapper[4524]: E1030 20:03:45.624945    4524 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-fht7f" containerName="extractor"
Oct 30 20:03:46.353973 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:46.353942    4524 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-trmds/perf-node-gather-daemonset-wzfm7"
Oct 30 20:03:47.220737 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:47.220704    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-2ltm4_741a2f85-9f59-443e-909f-ff76c3816645/discovery/0.log"
Oct 30 20:03:47.864244 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:47.864210    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-678bf485d7-kpxzm_b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2/kube-auth-proxy/0.log"
Oct 30 20:03:47.903820 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:47.903794    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-6hpnk_bb42d5ce-ed59-413b-9296-d2a0c09f8499/istio-proxy/0.log"
Oct 30 20:03:47.942709 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:47.942678    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-bt7bg_cb7c8396-fff1-47c2-9def-f721fb1eaeb4/istio-proxy/0.log"
Oct 30 20:03:48.163878 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:48.163786    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-rc4rt_64b8f203-6e98-4ce3-adea-8686640b74e8/istio-proxy/0.log"
Oct 30 20:03:51.283684 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:51.283597    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-2ltm4_741a2f85-9f59-443e-909f-ff76c3816645/discovery/0.log"
Oct 30 20:03:51.321288 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:51.321260    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-678bf485d7-kpxzm_b95fd459-3c9c-4a5a-9a4f-6c08116e6ca2/kube-auth-proxy/0.log"
Oct 30 20:03:51.346584 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:51.346561    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-6hpnk_bb42d5ce-ed59-413b-9296-d2a0c09f8499/istio-proxy/0.log"
Oct 30 20:03:51.381487 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:51.381445    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-bt7bg_cb7c8396-fff1-47c2-9def-f721fb1eaeb4/istio-proxy/0.log"
Oct 30 20:03:51.453375 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:51.453343    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xtj4p_7411c16b-533b-4152-bbe6-52262afa37da/serve-healthcheck-canary/0.log"
Oct 30 20:03:51.498882 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:51.498849    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-855f9574cf-rc4rt_64b8f203-6e98-4ce3-adea-8686640b74e8/istio-proxy/0.log"
Oct 30 20:03:52.045318 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:52.045291    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fht7f_c4b9f9bd-6097-4524-9c86-44c7f484f278/kube-rbac-proxy/0.log"
Oct 30 20:03:52.076751 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:52.076727    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fht7f_c4b9f9bd-6097-4524-9c86-44c7f484f278/exporter/0.log"
Oct 30 20:03:52.107715 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:52.107685    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fht7f_c4b9f9bd-6097-4524-9c86-44c7f484f278/extractor/0.log"
Oct 30 20:03:52.370476 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:52.370446    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_periodic-gathering-zs5lf-ptq4h_579247bb-977d-4c89-99cd-2831cb2ed824/insights-gathering/0.log"
Oct 30 20:03:55.044405 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:55.044370    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-6cf687fb7-fx8th_669db161-f0cc-42e6-b9f6-8cff97eb4891/manager/0.log"
Oct 30 20:03:55.071142 ip-10-0-137-200 kubenswrapper[4524]: I1030 20:03:55.071112    4524 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-86vsp_67bbe241-581a-468f-a88f-1b48afdfdc1d/manager/0.log"